President, @LawReformInst; previously executive director @uniformlaws, attorney @StateDept
McLean, Virginia
Joined January 2016
- Tweets851
- Following720
- Followers571
- Likes8.8K
Pinned Tweet
New from Dan Crane and me in @just_security: an explanation of why the bipartisan Collaboration on Adversarial Threats and Security Risks Act is necessary.
Tim Schnabel retweeted
Here's the letter sent by the White House on Friday laying out its rationale for not sharing information on the AI framework, as well as the list of documents it produced as a result of the lawsuit
protectdemocracy.org/wp-cont…
🚨 INFOHAZARD 🚨
In this Statecraft episode, @calebwatney makes a compelling argument that you, yes you, should quit your job. Be ye warned.
Co-hosted by the singular @LeahLibresco statecraft.pub/p/how-to-give…
Anybody else heading to Philly today for the @AmLawInst meeting on civil liability for AI? Good opportunity to heckle @ketanrama
Tim Schnabel retweeted
If it’s true that China has switched to distilling from OpenAI models because OpenAI hasn’t implemented as strong anti-distillation countermeasures as Anthropic has, then OpenAI and Google should implement comparable measures to Anthropic’s. If the government needs to give the AI labs a narrow anti-trust waiver to allow them to coordinate on this specific problem set to allow that to happen, it should do so. This seems like it would be in all the companies’ interests.
It would actually be a really good news story if Anthropic’s anti-distillation measures are working! At the beginning of this year, it was not at all obvious whether US labs could make it substantially harder for Chinese labs to distill US models simply by implementing stronger internal policies and security measures. But the US ability to defend against distillation attacks will be defined by the frontier lab with the weakest anti-distillation policy, not the strongest one.
SITUATION EXPLAINED: Anthropic cut off Chinese labs from distilling Claude, so they reportedly switched to OpenAI's GPT-6 Astra.
• Moonshot's Kimi K3.1 Pro and DeepSeek's V4.1-Flash 1017, both released this month, were reportedly trained on Astra's outputs, and neither lab has commented
• Distillation means training a smaller "student" model to copy a stronger "teacher" model's answers, and it works through a normal API without the teacher's weights
• In February, Anthropic said DeepSeek, Moonshot and MiniMax used about 24,000 fake accounts to run over 16 million chats with Claude
• Distillers mostly want Claude's step-by-step reasoning, and in Anthropic's own test, a model trained on it scored 80.4% on a coding benchmark vs 71.8% when trained on summaries
• Anthropic now hides that reasoning from distillers and bans their accounts, often within 2 hours of a campaign starting
• Anthropic reportedly told investors distillation made up 5% to 7% of its revenue, and it's walking away from that money to shut it down
• OpenAI said in September that it traced a distillation campaign that began in July to people linked to Moonshot
@theojaffee: "It's not all distillation. I think there's a little bit of cope going on here. They genuinely have amazing talent. They're genuinely able to work very well with very limited resources compared to what US labs have in terms of compute."
I love scheduled tasks in ChatGPT, but it's annoying when the system threatens to pause a task despite my reading the output every day.
Tim Schnabel retweeted
I should probably write something about this book tour, it's been fascinating to go on a bunch of podcasts and TV shows and see all the flavors of AI skepticism my fellow journalists have adopted.
Looks like the rumors were correct, re: Anthropic's founders retaining majority voting control.
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
Seems like the founders control 3 board seats, the LTBT controls 3, and the LTBT can make changes to the 7th board seat only if (non-founder) employees don't object? If that's right, it's very interesting-- effectively, the employees will retain veto rights over control.
Tim Schnabel retweeted
It's kinda interesting from a privacy perspective that we used to leave our names in the cards of every book we checked out, with a record we checked it out on a particular date and our signature, for anyone who later has the book to see the history of who checked it out.
Tim Schnabel retweeted
Unrelated to everything: as we are now a bit more than ten years into the deep learning revolution, we should be incredibly conscious of the following dynamic:
Step 1: A technology prototype based on a simple principle is demonstrated.
Step 2: The strongest argument that it won't work amounts to "nuh-uh, the problem has to be harder than that."
Step 3: The consequences for the prototype becoming a mature technology are so extreme that many people reject either the conclusions or the argument that the principle really works.
Step 4: As the tech advances, disbelief becomes entrenched. This creates significant resistance to seriously preparing for the impacts of the technology.
Step 5: The technology becomes mature and the consequences are increasingly realized. Because of entrenched disbelief over time, people didn't prepare enough.
For whatever next technology might follow this pattern, we need to aggressively push back on the disbelief from earlier on in the cycle. We cannot afford to let such short-sighted viewpoints derail society-wide preparedness again. The stakes and consequences are only getting higher.
Tim Schnabel retweeted
I suspect that this might just be a post-training RL artifact left over in the model. But we for sure need better honeypot evals for open source models to better understand potential backdoors or accidental risks like this one.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Anthropic also made disclosures about the antitrust risks it faces based on efforts to address "the responsible pace of AI development."
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
Very interesting that Anthropic highlights that its "future models ... may be able to produce export-controlled software," but they say nothing in that respect about current models, let alone export controlled "technology" or "technical data." How is it not a relevant risk that, at any time, BIS and DDTC could decide to enforce EAR and ITAR restrictions based on models being able to output information necessary for producing/using export-controlled items?
Also interesting that they flagged that "[i]n certain contexts, the provision of our models, platform, and applications may constitute export of defense articles or defense services subject to the ITAR, which can be exported only if we obtain required export licenses or other government authorizations." Glad to see they're not focused solely on the EAR... but still weird that they drew the line in this manner?
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.