she | endocrine disruptor, small molecule enjoyer, aspiring computer toucher. dm for unblock
london
Joined September 2023
- Tweets10.7K
- Following823
- Followers354
- Likes104K
man can you imagine how hard this would go as a three part documentary. we must make it to 2040
Ben Mann on why he left OpenAI, in a deposition ("Yann LeCun" is probably a mishearing of "Jan Leike")
![Q: And why did you subsequently leave OpenAI to join Anthropic? MS. MEHTA: [OpenAI confidentiality caution] but you can answer. THE WITNESS: When I was in sort of the last couple of months of my time at Anthropic, there were multiple instances where the safety team, which was the team that I was on, along with the other Anthropic cofounders, and the executive team, which at the time was led by Sam Altman, would disagree about our company strategy, and in particular, we felt that safety wasn't really considered to be a first-class priority. And I think there are many reasons that you could reasonably believe that, but we all believed that safety was the most important thing, which is not very surprising, given the team that we were on. And the other seven cofounders all decided to leave together. I wasn't involved in those discussions or decisions, and so they -- they sort of all left together. And before I left, Mira Murati, who, at the time, was, I believe, the CTO of the company…](https://pbs.twimg.com/media/HWo2HnCXMAAiQGI.png?name=orig)
ALT Q: And why did you subsequently leave OpenAI to join Anthropic? MS. MEHTA: [OpenAI confidentiality caution] but you can answer. THE WITNESS: When I was in sort of the last couple of months of my time at Anthropic, there were multiple instances where the safety team, which was the team that I was on, along with the other Anthropic cofounders, and the executive team, which at the time was led by Sam Altman, would disagree about our company strategy, and in particular, we felt that safety wasn't really considered to be a first-class priority. And I think there are many reasons that you could reasonably believe that, but we all believed that safety was the most important thing, which is not very surprising, given the team that we were on. And the other seven cofounders all decided to leave together. I wasn't involved in those discussions or decisions, and so they -- they sort of all left together. And before I left, Mira Murati, who, at the time, was, I believe, the CTO of the company…
![…offered for me to become the leader for the safety team, which, at the time, was headless because all the Anthropic cofounders, with the exception of me and Tom Brown, were safety team leads, and the only other one was Paul Christiano, who also decided to leave and start his own company. And that was actually a very difficult decision for me to make, whether I should stay and try to keep the torch of safety burning at OpenAI, or leave [audio interruption] and start from zero and work with these people, who I had … trusted very, very deeply, but … were starting with no funding and no code … and no resources, no real reputation… And, ultimately, I made the decision to leave because I had a conversation with one of the founders of Open Philanthropy project, who I deeply trusted, and he made an analogy …: when you're dating someone, either you can go for the person who looks good on paper or you can go for the person who you have chemistry with. And that just immediately made it became…](https://pbs.twimg.com/media/HWo2HnBWAAABv2_.png?name=orig)
ALT …offered for me to become the leader for the safety team, which, at the time, was headless because all the Anthropic cofounders, with the exception of me and Tom Brown, were safety team leads, and the only other one was Paul Christiano, who also decided to leave and start his own company. And that was actually a very difficult decision for me to make, whether I should stay and try to keep the torch of safety burning at OpenAI, or leave [audio interruption] and start from zero and work with these people, who I had … trusted very, very deeply, but … were starting with no funding and no code … and no resources, no real reputation… And, ultimately, I made the decision to leave because I had a conversation with one of the founders of Open Philanthropy project, who I deeply trusted, and he made an analogy …: when you're dating someone, either you can go for the person who looks good on paper or you can go for the person who you have chemistry with. And that just immediately made it became…
ALT …clear that I wanted to go to Anthropic and work with all these people who I really loved working with and deeply, deeply trusted, instead of playing a political game for the next n years of my career. And on top of that, I also had a conversation with Yann LeCun, who Dario had hired but who hadn't started working at OpenAI yet, and I told him, If you come to this company and you become the lead of the safety team, you're going to be playing a political game. You're not necessarily going to get to focus on the safety research that you're dedicated to doing. Are you okay with that? And it was because he said yes that I felt comfortable sort of following my heart and going to Anthropic. BY MR. MILLER: Q: Are you -- withdrawn. Do you consider yourself a cofounder of Anthropic? MS. MEHTA: Objection to the form. THE WITNESS: So from a corporate governance standpoint, I am not a cofounder of Anthropic. However, between me and the other cofounders, de facto, I am, and I've played that…
saliva retweeted
I know this is a predictable opinion from me but I don't agree with the strong emotional reaction against the anti-data-center movements that I've seen this week.
As I see it, algorithmic progress means that whatever you would be able to do with more and bigger data centers in 2 years, you will be able to do with the data centers we have today in 4 years - probably sooner. So nothing remotely apocalyptically bad will happen from even extreme levels of success by anti-data-center people.
Meanwhile we need all the time we can get to better harden the world against both already-visible cybersecurity issues and any bio and other risks soon to come. Science applications are bottlenecked by many things other than AI, so slowing AI down by one year will slow those down by much less than one year.
Given this, a slowdown in compute hyper-scaling seems quite net good? Notice this is without any controversial "doomer arguments", RSI etc. Even if net-bad, it's certainly not worse than like 10+ other political things.
saliva retweeted
I find Anthropic's RSP frustratingly difficult to interpret, and I'm not an expert in any of this (I'm just reading it over the weekend). That said, it seems to me like we should expect (a) external review of the next Risk Report and (b) a public analysis of Model 4's risks by late November. This is because Model 4 seems likely to be Anthropic's first "highly capable" model, and substantially riskier than publicly analysed models.
I'd love corrections/pushback! Here's my current understanding:
1. Anthropic's RSP defines a category of "highly capable" models. A model is "highly capable" if it crosses the AI R&D threshold they define in Section 1.
2. There are 2 ways to cross this threshold, and either is (individually) sufficient:
2a. If Anthropic's models could fully substitute for its entire research staff at competitive cost (within a factor of 5). This isn't currently the case.
2b. If Anthropic observes or expects double the rate of progress in aggregate AI capabilities, compared both to the rate it would expect and to the fastest sustained rate observed without significant AI contributions, over at least three model generations, and it's plausible this doubling is substantially attributable to automated research or engineering. A footnote clarifies "double" as two years of baseline progress in one year.
3. It seems plausible that condition 2b has been met by Anthropic's most capable internal model, "Model 4" (as described by METR here: metr.org/blog/2026-11-05-ant…)
3a. METR estimates that Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.
3b. METR also note that the indicators Anthropic use in their redacted risk report imply an acceleration rate of 1.97x, and their own analysis is that estimates between 1.3 and 2.8x are reasonable.
3c. Anthropic's criterion for crossing their AI R&D threshold is satisfied by expecting doubling, point estimates for both are around 2 with big error bars, and Anthropic's own private assessment says exceeding 2 is "plausible." I think that a reader trying in good faith to assess whether this AI R&D threshold has been crossed would conclude that it has been.
3d. If this AI R&D threshold has been met, then the models causing such AI R&D acceleration are (by the RSP's definition) "highly capable".
4. Anthropic makes several commitments about what they'll do once they have a highly capable model. The commitments they make relative to their competitors are:
4a. If they're "in the lead", i.e. they have strong evidence that no other competitors have a "highly capable" model: delay AI development and deployment until they have a strong argument that catastrophic risk is contained
4b. If competitors have, or will soon have, a "highly capable" model, and competitors can make such strong arguments: meet or exceed those competitors' risk reduction efforts; delay development and deployment as necessary to do so
4c. If they have strong evidence that competitors have better risk mitigations, which Anthropic could implement at comparable costs: implement them
5. Separately, Anthropic commit to a full external review by an LTBT-approved reviewer, with public commentary within 30 days, of any Risk Report that covers highly capable models and is significantly redacted.
5a. The next Risk Report is due between November and February, 3-6 months after the previous report.
6. Anthropic also makes a separate commitment, which is not in terms of "highly capable" models: within 30 days of determining that an internally deployed model's automated R&D or misalignment risks "significantly exceed" those of any model Anthropic has publicly analysed, Anthropic commit to publishing an analysis of that model's risks.
7. I think that if Anthropic's Model 4 is highly capable, then its risks significantly exceed previous models.
7a. This isn't implied by Anthropic's RSP, this is me trying in good faith to interpret the RSP.
7b. Anthropic generally uses "highly capable" as the trigger for their other substantial commitments (to the extent that they make any particularly substantial commitments) in their RSP.
7c. In some sense, this distinction is arbitrary: Anthropic have picked some point at which models are great at AI R&D and called that line "highly capable".
7d. But in general, this "highly capable" category triggers step changes in how Anthropic responds to risks. I think it would be very strange, and would smell of motivated reasoning, if Anthropic thought their most recent model was "highly capable" but didn't have risks that significantly exceeded those of the models they've published analyses for.
7e. So if Model 4's risks do "significantly exceed" those of the models Anthropic has publicly analysed the risks from, Anthropic should publish an analysis of its risks within 30 days of determining this "significantly exceeding" criterion has been met.
8. If this is right, when should we expect a public report?
8a. Based on METR's report, Model 4 finished training in September–October and was available to most Anthropic staff in October.
8b. METR delivered their assessment of AI R&D acceleration, which found ~2x the rate of AI capabilities development vs a March 2024 - October 2025 baseline, to Anthropic on October 22.
8c. Let's say that Anthropic had no idea how good Model 4 was at AI R&D until they read METR's final report. My interpretation is that from October 22, Anthropic had reason to think that their most capable model was meeting their "highly capable" threshold defined in their RSP.
8d. Again, assuming that that model's risks therefore significantly exceed those of previously publicly analysed models, which seems plausible, Anthropic should publish a report within 30 days.
8e. So it seems like we should expect a report by the 21st November on Model 4's risks.
There are a lot of different numerical estimates here and different deadlines, and I think it's fairly likely I'm making a mistake somewhere! I've tried to lay out my thinking very clearly so people can easily correct me. (Usually I'd just post this on slack, but it seems better to be wrong in public?)
saliva retweeted
if you don't trust your model to operate gun drones in your office, you shouldn't trust it to scale up RSI to superintelligence.
saliva retweeted
Should you donate $10,000 to vaccinate 2000 infants, or give TB medicine to 1000 adults? Wrong question - you can't judge lives like feathers on a scale. Priorities should be decided on democratically. You should buy one-fiftieth of a Patriot missile for war with Iran.
saliva retweeted
@AnthropicAI: what the fuck? The compute allocation assessment is the thing we've been pushing for the last few months. It's very important, because it is the foundation on which pacing verification depends. "to keep the focus on risk assessment" is a non-answer, a non-excuse. Do you want to pace or not?
saliva retweeted
We cut through the mystery around interpretability in training.
A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities.
We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment.
But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵
whatever edge prediction markets had over the models almost certainly did not come from randoms' gut calls or whatever. what a disgusting grift
Last night, the power to decide what America sees changed hands.
I have spent a ton of time in DC over the last few months. The elites were certain the markets were off. I heard many variants of “no chance markets are right about X” and “I’ve been in politics for a decade. ‘These people’ don’t really know much.”
Last night, ‘these people’ beat the elites.
Prediction market favorites won 479 of the first 487 races. That’s 98%.
The elites live in a bubble. They theorize, write, ideate, and mistake credentials for an understanding of lives they’ve never lived. They don’t see and feel what the everyday man, on the ground, sees and feels.
Knowledge is not exclusive to the elites but scattered across millions of people.
Someone knows their hometown better than any national pundit. Someone has followed a local race passionately for months. Someone has followed a local race dispassionately for months. Someone notices a shift in their community before it makes the headlines. Someone is shaped by an incident at work. Someone is shaped by what’s getting harder or easier for their families.
Each holds a piece of the picture that no expert can see in full.
And for the longest time, that knowledge had no way to reach the rest of the world.
For generations, election night meant turning on the TV and waiting for a handful of people to tell you what happened.
Prediction markets change that.
You don't need a title, a large following, or an editor to publish you. All you need is a perspective and the conviction to back it.
That changes who holds power over information.
Anyone can see the market. Anyone can contribute to it. Anyone can question it. No title gives you the final word. No gatekeepers can filter your view.
Sometimes the signal can be wrong. But anyone can challenge the consensus and improve it.
And when people consolidate their knowledge together, something remarkable happens: millions of individual perspectives become a signal that we all can learn from.
Over the last few months, the people showed up on Kalshi to tell the world what they were going to do.
Last night, the megaphone was passed to them.
worst guy from your high school is commenting under reels titled Anthropic interview question number 52
saliva retweeted
i see a lot of people saying holy shit about this
i am not surprised
(i have receipts, i'm not one of those ppl who pretend to never be surprised but never predict the outcome beforehand)
even the most trustworthy-and-sympathetic-to-AI parties (ie my org, anima labs) struggle even w/ claudes sandbagging on some safety and interp related research; they are probably worse for anyone else. and astra is way worse. we don't even try to rope astra into most safety research as it seems like something that's obviously going to be a struggle
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
saliva retweeted
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
saliva retweeted
I'm okay talking about margins when it comes to Abdul El-Sayed. I would've liked that race to be called sooner too.
But yall need to take Troy Jackson's name out of your mouths. Susan Collins is a human institution. Troy, after the most nightmarish cycle ever, just beat God.
