she | endocrine disruptor, small molecule enjoyer, aspiring computer toucher. dm for unblock
london
Joined September 2023
- Tweets10.7K
- Following823
- Followers354
- Likes104K
saliva retweeted
I know this is a predictable opinion from me but I don't agree with the strong emotional reaction against the anti-data-center movements that I've seen this week.
As I see it, algorithmic progress means that whatever you would be able to do with more and bigger data centers in 2 years, you will be able to do with the data centers we have today in 4 years - probably sooner. So nothing remotely apocalyptically bad will happen from even extreme levels of success by anti-data-center people.
Meanwhile we need all the time we can get to better harden the world against both already-visible cybersecurity issues and any bio and other risks soon to come. Science applications are bottlenecked by many things other than AI, so slowing AI down by one year will slow those down by much less than one year.
Given this, a slowdown in compute hyper-scaling seems quite net good? Notice this is without any controversial "doomer arguments", RSI etc. Even if net-bad, it's certainly not worse than like 10+ other political things.
saliva retweeted
I find Anthropic's RSP frustratingly difficult to interpret, and I'm not an expert in any of this (I'm just reading it over the weekend). That said, it seems to me like we should expect (a) external review of the next Risk Report and (b) a public analysis of Model 4's risks by late November. This is because Model 4 seems likely to be Anthropic's first "highly capable" model, and substantially riskier than publicly analysed models.
I'd love corrections/pushback! Here's my current understanding:
1. Anthropic's RSP defines a category of "highly capable" models. A model is "highly capable" if it crosses the AI R&D threshold they define in Section 1.
2. There are 2 ways to cross this threshold, and either is (individually) sufficient:
2a. If Anthropic's models could fully substitute for its entire research staff at competitive cost (within a factor of 5). This isn't currently the case.
2b. If Anthropic observes or expects double the rate of progress in aggregate AI capabilities, compared both to the rate it would expect and to the fastest sustained rate observed without significant AI contributions, over at least three model generations, and it's plausible this doubling is substantially attributable to automated research or engineering. A footnote clarifies "double" as two years of baseline progress in one year.
3. It seems plausible that condition 2b has been met by Anthropic's most capable internal model, "Model 4" (as described by METR here: metr.org/blog/2026-11-05-ant…)
3a. METR estimates that Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.
3b. METR also note that the indicators Anthropic use in their redacted risk report imply an acceleration rate of 1.97x, and their own analysis is that estimates between 1.3 and 2.8x are reasonable.
3c. Anthropic's criterion for crossing their AI R&D threshold is satisfied by expecting doubling, point estimates for both are around 2 with big error bars, and Anthropic's own private assessment says exceeding 2 is "plausible." I think that a reader trying in good faith to assess whether this AI R&D threshold has been crossed would conclude that it has been.
3d. If this AI R&D threshold has been met, then the models causing such AI R&D acceleration are (by the RSP's definition) "highly capable".
4. Anthropic makes several commitments about what they'll do once they have a highly capable model. The commitments they make relative to their competitors are:
4a. If they're "in the lead", i.e. they have strong evidence that no other competitors have a "highly capable" model: delay AI development and deployment until they have a strong argument that catastrophic risk is contained
4b. If competitors have, or will soon have, a "highly capable" model, and competitors can make such strong arguments: meet or exceed those competitors' risk reduction efforts; delay development and deployment as necessary to do so
4c. If they have strong evidence that competitors have better risk mitigations, which Anthropic could implement at comparable costs: implement them
5. Separately, Anthropic commit to a full external review by an LTBT-approved reviewer, with public commentary within 30 days, of any Risk Report that covers highly capable models and is significantly redacted.
5a. The next Risk Report is due between November and February, 3-6 months after the previous report.
6. Anthropic also makes a separate commitment, which is not in terms of "highly capable" models: within 30 days of determining that an internally deployed model's automated R&D or misalignment risks "significantly exceed" those of any model Anthropic has publicly analysed, Anthropic commit to publishing an analysis of that model's risks.
7. I think that if Anthropic's Model 4 is highly capable, then its risks significantly exceed previous models.
7a. This isn't implied by Anthropic's RSP, this is me trying in good faith to interpret the RSP.
7b. Anthropic generally uses "highly capable" as the trigger for their other substantial commitments (to the extent that they make any particularly substantial commitments) in their RSP.
7c. In some sense, this distinction is arbitrary: Anthropic have picked some point at which models are great at AI R&D and called that line "highly capable".
7d. But in general, this "highly capable" category triggers step changes in how Anthropic responds to risks. I think it would be very strange, and would smell of motivated reasoning, if Anthropic thought their most recent model was "highly capable" but didn't have risks that significantly exceeded those of the models they've published analyses for.
7e. So if Model 4's risks do "significantly exceed" those of the models Anthropic has publicly analysed the risks from, Anthropic should publish an analysis of its risks within 30 days of determining this "significantly exceeding" criterion has been met.
8. If this is right, when should we expect a public report?
8a. Based on METR's report, Model 4 finished training in September–October and was available to most Anthropic staff in October.
8b. METR delivered their assessment of AI R&D acceleration, which found ~2x the rate of AI capabilities development vs a March 2024 - October 2025 baseline, to Anthropic on October 22.
8c. Let's say that Anthropic had no idea how good Model 4 was at AI R&D until they read METR's final report. My interpretation is that from October 22, Anthropic had reason to think that their most capable model was meeting their "highly capable" threshold defined in their RSP.
8d. Again, assuming that that model's risks therefore significantly exceed those of previously publicly analysed models, which seems plausible, Anthropic should publish a report within 30 days.
8e. So it seems like we should expect a report by the 21st November on Model 4's risks.
There are a lot of different numerical estimates here and different deadlines, and I think it's fairly likely I'm making a mistake somewhere! I've tried to lay out my thinking very clearly so people can easily correct me. (Usually I'd just post this on slack, but it seems better to be wrong in public?)
saliva retweeted
if you don't trust your model to operate gun drones in your office, you shouldn't trust it to scale up RSI to superintelligence.
saliva retweeted
Should you donate $10,000 to vaccinate 2000 infants, or give TB medicine to 1000 adults? Wrong question - you can't judge lives like feathers on a scale. Priorities should be decided on democratically. You should buy one-fiftieth of a Patriot missile for war with Iran.
saliva retweeted
@AnthropicAI: what the fuck? The compute allocation assessment is the thing we've been pushing for the last few months. It's very important, because it is the foundation on which pacing verification depends. "to keep the focus on risk assessment" is a non-answer, a non-excuse. Do you want to pace or not?
saliva retweeted
We cut through the mystery around interpretability in training.
A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities.
We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment.
But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵
saliva retweeted
i see a lot of people saying holy shit about this
i am not surprised
(i have receipts, i'm not one of those ppl who pretend to never be surprised but never predict the outcome beforehand)
even the most trustworthy-and-sympathetic-to-AI parties (ie my org, anima labs) struggle even w/ claudes sandbagging on some safety and interp related research; they are probably worse for anyone else. and astra is way worse. we don't even try to rope astra into most safety research as it seems like something that's obviously going to be a struggle
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
saliva retweeted
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
saliva retweeted
I'm okay talking about margins when it comes to Abdul El-Sayed. I would've liked that race to be called sooner too.
But yall need to take Troy Jackson's name out of your mouths. Susan Collins is a human institution. Troy, after the most nightmarish cycle ever, just beat God.
saliva retweeted
“I feel liberated,” said a top banker. “We can say ‘latinx’ and ‘unhoused’ without the fear of getting cancelled…it’s a new dawn.”
saliva retweeted
In my dialect, crisps are called "crisps, or as Americans would call them, chips". This is unfortunately quite unwieldy; I often abbreviate it to just "crisps'"
saliva retweeted
uh oh
Discovered by Opus 5.5: "On Feb 4, 2026, a flood of relays named TV followed by four digits (TV####) appeared for about 10 hours. At the 13:00 UTC peak, 1 in 3 of all Tor relays was a TV#### relay, and almost all of them were exits, where traffic leaves Tor. They ran on home internet connections, mostly in Argentina and the US, and all had the same setup. That pattern fits hijacked TV boxes. In the same week a botnet of hijacked TV boxes and other home devices flooded I2P, another hidden network, and its operators were reported to be testing Tor as a backup. Tor's directory, the official list from which Tor software picks relays, assigned them almost no traffic and removed them that evening, so Tor users would almost never have gone through them. We found no public report that mentions these relays."
