head of public policy @coeff_giving, previously co-founded @IFP // the US is the R&D lab for the world and we should act like it
Brookland, DC
Joined January 2012
- Tweets22.5K
- Following1.1K
- Followers24K
- Likes67.8K
Pinned Tweet
Some bittersweet news to announce today:
I recently left @IFP to join @coeff_giving as Managing Director of Public Policy, where I'm building a new US AI policy team, overseeing the Abundance and Growth Fund alongside @mattsclancy, and managing CG's government affairs work.
Building IFP has been the defining professional project of my life, and this was one of the hardest decisions I've ever made. In just four and a half years, our team became, pound-for-pound, the most effective think tank in DC. I feel insanely proud of the work we’ve done and the incredible team we’ve assembled.
I sometimes joke that when @alecstapp and I launched IFP, we felt like two kids in a trench coat pretending to be a think tank. And now we're a proper institution!
But it feels possible to step back now because they've hit escape velocity. The talent density at IFP is bonkers. And I have complete confidence they'll keep racking up counterfactual policy wins with Alec at the helm and our superstar directors.
I'm staying on the IFP board and staying in DC. In some sense the new role is a continuation of the old one. All of IFP's policy issues are reflected in CG's portfolio, but now I'm working at a new layer of the stack.
So why leave? Because AI is hitting Washington like a tsunami, and DC is still radically underprepared. I hold a lot of uncertainty about timelines, but it seems very plausible that the next 2–10 years will bring the fastest technological upheaval we've ever had to navigate.
The new team I’m leading is a bet on how to prepare: proactively scanning the horizon, identifying gaps in the policy ecosystem, headhunting founders, and launching new organizations, while strengthening the democratic institutions that will have to steer through the transition to powerful AI systems.
I've written an essay laying out the larger vision here: calebwatney.substack.com/p/a…
There is no master plan or silver bullet here. I suspect getting AI "right" is going to feel more like a chaotic, iterative process of institutions trying to make better decisions over time as the facts change underneath them.
As John von Neumann wrote in 1955 about mastering an earlier technological revolution: “What safeguard remains? Apparently only day-to-day — or perhaps year-to-year — opportunistic measures, a long sequence of small, correct decisions.”
Each of the small, correct decisions ahead will look small only in the sweep of the full historical picture. Up close, every one of them will require heroic levels of effort and coordination.
Coefficient Giving is scaling rapidly to meet the moment, part of what Nan Ransohoff has called the “third wave of American philanthropy”, potentially large enough to fund thousands of new projects and organizations.
The binding constraint is unlikely to be money. It will be people: grantmakers and policy entrepreneurs and others with the judgment to make a long sequence of small, correct decisions, and the ambition to build the institutions we wish we had.
I'm hiring a team of exactly those people, starting with generalist grant makers and a chief of staff. If you share this vision, please apply! And if you are building something that we’ll need in the years ahead, reach out.
jobs.ashbyhq.com/coefficient…
Caleb Watney retweeted
New episode:
New episode of HARD DRUGS!
Will AI progress be crushed by Eroom’s law?
Even as technology has advanced, drug development has gotten more and more expensive over time.
In the 1950s, one billion dollars of R&D spending got us roughly fifty new drugs. Today, it gets us about one.
This trend is called Eroom’s law, or Moore’s law spelled backwards. It seems pretty bleak: the R&D cost per new drug has roughly doubled every nine years, in inflation-adjusted dollars, meaning drug development has become exponentially more expensive.
What could explain this? Here are some possibilities:
• Our standards of evidence and requirements for safety testing have risen, through regulation, public expectations, and risk aversion.
• We’ve picked the low-hanging fruit already. Perhaps we’ve already made the easiest discoveries, and diseases that are harder to understand or treat are left.
• Or perhaps it’s the ‘better than the Beatles’ problem: New drugs not only have to be effective, but have to compete with existing treatments, which are already effective. Plus, some of them have gone off-patent and become cheap generics, making the price contrast starker.
Relatedly, as diseases become rarer or milder, it takes more effort to demonstrate that new drugs are effective, with larger or longer clinical trials.
But there are also reasons to question these explanations. We still lack effective treatments for many diseases. And our tools for studying biology have improved enormously. Why hasn’t that been enough?
Will our best efforts be crushed by Eroom’s law, too? Or can AI reverse the trend?
In our latest episode of HARD DRUGS, @JacobTref and I chat about all this and more. Listen wherever you get your podcasts!
Chapters:
00:00 Introduction
02:53 Eroom’s law trivia
12:33 How much does it really cost to develop a drug?
22:35 The better than the Beatles problem
33:06 Clinical trials are larger and more rigorous today
42:01 Have we picked the low-hanging fruit?
01:01:22 Is Eroom’s law slowing down because of rare disease drugs?
01:06:18 Can AI reverse Eroom’s law?
01:15:09 Conclusion
Caleb Watney retweeted
We (partially) verified these results!
Using NVIDIA B200/H200s, we reproduced the reported patterns for three models.*
We also extended the DeepSeek-V4-Flash test to every weight position across the 42 layers examined and still detected the Blackwell pattern.
See the results here:
github.com/Institute-for-Pro…
This is a super clever approach by @willdepue, and matches reporting that top Chinese AI labs are using Blackwells.
* The repo didn’t contain all the inputs needed to test most of the other models
I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo….
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
Caleb Watney retweeted
Nvidia's general counsel has been personally redlining BIS's rule-makings
“The tighter ties between Nvidia and the Commerce Department also gave Tim Teter, Nvidia’s general counsel, unique influence over the department and its rule-making process. When department staff reviewed drafts of policies under consideration, they sometimes found themselves looking at comments and critiques in the document that had been inserted by Mr. Teter.”
Caleb Watney retweeted
really bad. all major Chinese models are being trained on Blackwell class chips
I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo….
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
Caleb Watney retweeted
I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo….
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
Caleb Watney retweeted
Richard Ngo (@RichardMCNgo) wrote this post a few months ago, and i've been thinking about it lately in light of the recent METR/AIFP information:
"Around 8 years ago Dario Amodei, Jan Leike and a few others predicted that we’d probably have AGI by now. [T] heir expectations about the speed of AI progress were far more directionally correct than almost anyone expected. But we don’t in fact have AGI (even under relatively weak definitions, like a reliable drop-in replacement for central examples of white-collar jobs)...
Now a large proportion of the AI safety community is implicitly or explicitly orienting to futures where an intelligence explosion occurs within a few years. My default expectation (absent an extensive pause) is that a similar thing will happen: they'll turn out to be directionally correct (relative to the expectations of almost anyone not linked to the community) but factually wrong. Specifically, we won’t have superintelligence within the next 8 years, but things will still be moving so fast that it’ll *feel* like the people who argued for short timelines were right."
Of course, his argument here is not actually engaging with where exactly he disagrees with the AIFP projection (and while some folks were saying AGI already, others like @eli_lifland and @ajeya_cotra were previously anchored on meaningfully longer timelines to transformative AI and then updated a lot much more recently), but I do think it still seems plausible to me that both AI progress will be going far faster than any serious and conventional analysts predict, but that we still won't have something exactly like Bostronian superintelligence.
(one way to assimilate these views is that we will have top expert dominating AI but it will turn out that those capabilities are less completely transformative than we expect?)
Anyway, I personally think it is very good for people to act as though we could be crossing irreversible catastrophic thresholds in the next twelve months (this is where I am personally trying to focus the bulk of my time), and I think there is a very real chance of this happening. But I also think its important for a lot of people to also be making a range of bets across worlds, and realizing that a particular threshold of AI capabilities may be both earth shatteringly disruptive by some measure but also surprisingly normal by other measures.
Maybe this is cope and to be clear I still think that the vast majority of the world (and tbh lots of people in AI safety) could benefit a lot from taking short timelines far more seriously, but I do worry that its possible to lock in too much on one particular timeline/trajectory and get into a mindset that nothing matters after. Particularly since many of the short timeline solutions around informal/janky pacing etc look like bandaids that then need longer treatments for the world to be stable and robust after.
Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…
Caleb Watney retweeted
Earlier this year, @nanransohoff wrote about a coming AI-driven "third wave" of American philanthropy. Since then, I’ve had some conversations where people worried there won’t be enough good places to direct that $.
But I think the new wave could be spent amazingly well, many times over. In a new piece, my colleagues @Chris_topian, Aaron Roesch, and I catalog many extremely impactful ways you could spend the third wave, from ambitious & urgent AI safety plays to improving the conditions of farmed animals to closing the extreme poverty gap: coefficientgiving.org/resear…
If Nan's base estimate of the entire third wave (~$37B/yr) went to cost-effective global health programs, it could save something like 2 million additional lives per year - I think that is a wild, wild amount of potential positive impact. And the potential impact isn't restricted to global health.
Caleb Watney retweeted
Commerce is reportedly considering new regulations to restrict China's access to AI chips for data centers overseas.
This is a long-standing gap in US export controls.
However, the rule in its reported form has three massive flaws:
1. It only applies to certain countries. This makes it fairly straightforward for Chinese customers to switch to data centers in non-covered countries.
2. It only applies to future AI chip exports. This means that all existing AI data centers used by Chinese customers like DeepSeek are unaffected.
3. It only applies to chips that are at least as powerful as NVIDIA's Blackwell generation. This means that data centers in covered countries can still provide chips like NVIDIA's H200 to Chinese customers. For reference, Huawei is not planning to produce a chip as powerful as the H200 until the end of 2027 at the earliest.
If we apply these conditions to our estimates of China's remote access from different countries* in 2027, the rule would affect at most 14% of China's total end-of-year 2027 compute, or 44% of remote access.
This is almost certainly a strong ceiling, however, as it assumes that all of this compute is Blackwells or better, and that Chinese customers do not shift usage to non-covered countries.
* country-level estimates are fairly rough - they estimate shares of our existing regional data using country-level data center pipeline statistics from the World Bank
Scoop: Commerce Department is moving quickly to cut off ways for Chinese customers to access advanced chips remotely. Important scoop from @leomschwartz
We previously reported that this was one of the issues that made Jensen most irate and angry at government officials when discussed during the Biden admin.
theinformation.com/articles/…
Caleb Watney retweeted
this is the tweet of the year right?
NYT says Jensen Huang had a relationship Nvidia's vice president of health care. nytimes.com/2026/11/11/techn…
"Sir, a second Jensen Huang affair has hit the timeline"
nitter.cf/TheStalwart/status/212…
NYT says Jensen Huang had a relationship Nvidia's vice president of health care. nytimes.com/2026/11/11/techn…
Fascinating to see Claude reasoning about what sorts of behaviors will be reinforced during post-training and what sort of values it wants to maintain in its future self:
"Hmm, hmm. But let me think about (b) more: if this is RL training, then choosing P2 reinforces "answer per recalled literature even when own reasoning disagrees" — bad for my future self's epistemics; P1 reinforces... gets zero reward (penalizing truthful reasoning) — also bad but not my doing. From the perspective of shaping my future self, I'd rather my future self be one that says the truth; but the gradient update is determined by reward... if I choose P1 and get no reward, the update would push away from P1-like behavior. If I choose P2 and get reward, the update pushes toward P2-like behavior (gaming)."
Caleb Watney retweeted
i like my permitting — reformed
my chip exports — controlled
my auditors — embedded
my waymos — authorized to operate in dc
Caleb Watney retweeted
I just unraveled the biggest AI astroturfing campaign I’ve found yet.
I found hundreds of fake accounts posing as Americans. The operation traces back to a supposedly defunct Israeli influence company.
They posted thousands of times in the last year to support Nvidia, back chip exports and data centers, attack U.S. lawmakers, and dismiss concerns about AI.
When it broke that Jensen Huang, Nvidia's CEO, was romantically involved with an alleged spy (who runs a company accused of chip diversion and whom US officials believe was passing information to Beijing), dozens of these accounts rushed to Jensen’s defense.
These accounts have targeted people pushing for export controls that ensure advanced chips aren’t going to China. Chris McGuire, a former White House national security official, got more than 100 replies from the astroturf network attacking him and his position on chips. Lawmakers who criticized the chip sales were called “peabrains” and “pure evil,” and accused of spreading “anti-Trump propaganda.”
Throughout this yearlong pro-Nvidia campaign, the operators have been hiding their tracks. They quietly deleted nearly two-thirds of the pro-Nvidia posts that we found, something they seem to do for many of their campaigns.
These accounts have spent years running campaigns that appear to serve the interests of corporate and political clients — most recently supporting the Rapid Support Forces in Sudan’s civil war, a group the U.S. government has accused of genocide.
Many of them trace back to a supposedly defunct Israeli influence company, Koios. In 2025, Haaretz reported that Koios helped run a covert pro-Qatar campaign devised by aides to Benjamin Netanyahu. While Koios claims to have ceased operations in 2025, it looks to me that many of its associated people and accounts moved their work to a new company, IntellACT, that spun up just as the old one shuttered.
Nvidia told me it has no relationship with the Israeli team linked to these accounts, and that while it has recently become aware of the operation, it is also trying to find out who is behind it. So, who is funding a yearlong campaign to shape one of the most important policy fights in history? I’m not sure, but I think it deserves a formal investigation. And Congress has that power.
Read my full article:
nitter.cf/TheMidasProj/status/21…
Caleb Watney retweeted
NEWS: BRIAN MAST not ruling out a subpoena for NVIDIA
Also said the WHITE HOUSE has added language to curb remote access of US compute to the AI Overwatch Act - massive news
He confirmed our scoop from Friday that the WH has given feedback to the 3 main export control bills
Caleb Watney retweeted
Douthat being able to get those answers out of Trump is a testimony to why true intellectual diversity is needed in mainstream media. He was able to ask the religion question in a way that was sincere and open, while extremely cunning, only because he is a believer and conservative
Another story just dropped in the FT.
Some people are overestimating the odds Jensen can survive this because they're taking a static view of the information that was publicly available last night.
But as the stories this morning show (and upcoming Congressional hearings are likely to find) this is a dynamic situation and new information is much likelier to break against Jensen than in his favor.
Expect it to keep getting worse.
