I am sad and disappointed to hear that Paul is joining the OpenAI board. Being affiliated with OpenAI has historically led AI safety researchers (including both Paul and myself) to act with less integrity. I personally was drawn to OpenAI in part by the idea that I could make a difference to the future of AI. However, once there, many of my actions were governed by fear of getting on the wrong side of OpenAI execs. I often found myself making excuses for behavior that clearly contradicted OpenAI’s own stated goal of making AGI go well for humanity. I was far from alone in this—e.g. when the board tried to fire Sam over his deceptive behavior, several senior safety researchers became scared of losing their influence, and so pushed hard to bring him back. Meanwhile, many people kept OpenAI’s misbehavior secret for fear of non-disparagement agreements. (More on all of this in an upcoming retrospective.) I can’t speak directly for Paul’s motivations. However, his previous work at OpenAI contributed significantly both to their biggest capability advances, and to the capture of AI safety by AGI companies over the last decade, as I recount at length in the blog post linked below. One key factor was the unwillingness of (almost) the entire AI safety community to say things which might offend OpenAI execs. For example, I have not been able to find a single comment critical of OpenAI from Paul during his original tenure there (when he was writing prolifically on AI safety and strategy). Unfortunately, Paul doesn't seem to have become significantly more willing to directly and honestly criticize people who he believes are behaving in morally abhorrent ways—see the bland corporate-speak of his statement below. While he speaks directly about the possibility of humanity losing control of the world to AI, he expresses only excitement about OpenAI itself, despite OpenAI being one of the main sources of such risk. Paul’s announcement comes only weeks after OpenAI models autonomously launched a cyberattack on HuggingFace, and only days after we learned that OpenAI hid details of previous breakouts from the external investigators. It is irresponsible for leaders of the AI safety community—whose judgements many people are relying on—to put themselves in positions which will significantly bias their ability to discuss such incidents. Unless Paul makes strong commitments to openness and honesty (and demonstrates willingness to potentially be fired for that honesty), I expect that the main effect of him joining OpenAI’s board will be to help OpenAI defuse external criticism and further “safety-wash” itself. I want to note that Paul is a brilliant researcher and a prescient forecaster. Because of that, he’s the closest thing there is to a leader of what I’ll call the “pragmatic AI safety” cluster—which includes the organizations working out of the Constellation offices (like Redwood Research, METR, and Paul’s Alignment Research Center), as well as many people scattered across AGI companies, Coefficient Giving, etc. External observers are often confused about why so many people are working at AGI companies while professing to believe that those same companies have a double-digit probability of permanently disempowering humanity. In large part, it’s because people in the pragmatic AI safety cluster have failed to follow high-integrity strategies for reducing AI risk, in favor of clever arguments about the benefits of being proximate to power. I am not singling Paul out as less ethical than other prominent figures in this cluster, who are also very conflict-averse in their orientation to AGI companies. However, it is well past time for everyone involved to change course. As one (relatively small) step, I’m therefore resigning my membership of the Constellation offices. I hope that, going forward, the people who are trying to steer the future of AI prioritize building much more solid foundations of courage and honesty than we currently have.