Daniel Pryor

I’m an avid Economist reader, but this article is wide off the mark for me. It claims that distillation makes China’s AI progress dependent on America. ASML’s dominance of advanced lithography machines hasn’t exactly led to China giving up on its semiconductor indigenisation push, has it? News over the summer of the U.S. raising concerns that an EUV machine had been smuggled into China is but one example. We heard just yesterday that Chinese companies allegedly distilled from Astra amidst an Anthropic clamp down. If these companies weren’t getting value from distillation, why would they continue, despite having now been called out by the USG for doing it? China will never accept dependence on America, especially when it comes to AI. The article further claims that cutting off distillation would “set these companies back months, but no more.” The point is not the length of time but how much of a lead in capabilities this translates to. Recent IFP analysis suggests that distillation accounts for half of the capability gap and the U.S. lead would be 13 months without it. Picture the AI capabilities we had 13 months ago versus now (and that doesn’t even account for what is available internally) and then think about where we could be 13 months from now. The article further states that: “safeguards to restrain American models on risky subjects, such as bioweapons, already make Chinese models safer.” This assumes that the only way Chinese models can have dangerous capabilities is by copying them from American ones. It is trivial to fine-tune models on additional data which can enhance their dual-use capabilities conditional on the data being available. This also doesn’t account for custom harnesses which can be developed to surround these models which can enhance their capabilities further even without fine-tuning. Finally, the claim that “Distillation […] reduces the chance of catastrophe” fails to account for the evidence we already have of how models developed by Chinese companies using adversarial distillation are popping up in the PLA. We also know (thanks to Anthropic) that PLA-affiliated surveillance activity was detected because they queried a Chinese model that silently routed their query to Claude as part of a distillation program! If we had a war between the U.S. and China, how does having highly capable models integrated into PLA systems that benefitted from adversarial distillation reduce the chance of catastrophe? Yes distillation benefits the consumer but the downsides of China doing this in an industrial way clearly outweigh the benefits in my view. anthropic.com/threat-intelli… bloomberg.com/news/articles/… reuters.com/technology/deeps… anthropic.com/news/distillat… openai.com/index/estimating-… cset.georgetown.edu/publicat… jamestown.org/chinese-resear… dset.tw/en/research/distilla… ifp.org/how-to-increase-amer… cisa.gov/news-events/cyberse…