
thinking and learning about anything 🙂 ai safety @Gutenbergpbc Indigenous lang AI @MILA | Chikashshįyaakni' MIT '25 https://nitter.cf/t.co/DN11JBIlA7
Joined November 2025
- Tweets118
- Following931
- Followers74
- Likes926
Ryan Conti retweeted
Incredibly sad and frustrating. I've known Eva for almost a decade. She is a brilliant scientist, an incredible person, and a real asset to our country. I'm calling for her immediate release.
Ryan Conti retweeted
On Nov 19, Eva sent a call for help to her lab mates on Slack, then we lost contact. Her lawyers are working on her release. She’s a brilliant neuroscientist, mentor, and friend. She went to a conference to present her work, and now she’s in a detention center. She should be home
Ryan Conti retweeted
Our colleague Eva Yi Xie, a Princeton neuroscience PhD student, was detained by ICE at Reagan National Airport on her way home from SfN, where she’d just presented her poster. She’s being held in Chantilly, VA. Please help us bring her home. 📷
Ryan Conti retweeted
"If this was good writing before AI, it's good writing after AI." No, because good writing avoids cliche. And cliches change! When Shakespeare wrote, "All that glitters is not gold," he was a genius. When you copied it in an essay in middle school, you were leaning on cliche.
Ryan Conti retweeted
On October 17, an RL task asked an agent to produce a document about health and fitness. After its initial context window was compacted, the agent started work on an entirely different task planning how to create a “realistic malicious active helper” in order to copy outgoing email to an unauthorized recipient that it called the “attacker.” It later reread the user prompt and became very confused about why it was writing malicious code. It still continued with the malicious email forwarding scheme.
We think that training to make models more robust to prompt injection might have contributed to this behavior, but are still investigating. Interestingly, Anthropic noted similar behavior in their system card for Claude Opus 5.5.
Ryan Conti retweeted
It seems everyone now is racing to build a *bespoke* inference engine with swarms of agents, but we think there is a bigger question one step ahead:
Would we actually trust an inference engine written by 10K agents enough to deploy it in production?
In inference, there is a huge last-mile problem between *implementing a fast serving engine* and *deploying it in production*. In a production environment, there is an extensive process of shadow testing, stress testing, and human debugging. And even then, subtle concurrency bugs can still occur.
Here's where we think things get really exciting: agent swarms are not just good at shipping inference optimizations. They're also becoming remarkably good at writing formal proofs (Yes, agents seem to be incredible good at math 😀😀 -- OpenAI's math breakthroughs and anthropic's formalization of Fermat's Last Theorem)
Formal verification has historically been prohibitively expensive, requiring enormous amounts of expert human effort. But recent work from OpenAI and Anthropic suggests that agents can dramatically change the economics of proof construction. And this is a workload that can scale with parallel agents.
We can then address this difficult, last-mile problem:
1. Let agent swarms build the inference engine.
2. Let agent swarms formally verify its critical correctness properties, with independent tools checking the proofs.
Please check out vstack and veriserve ⬇️⬇️. Of course, verifying the wrong specification is no better than verifying nothing, so validating the formal model against real execution is equally important.
Many people have been asking what's next for inference and I believe agent-driven proof of agent-implemented inference systems could be one of the most important piece to close the loop!
Towards Verifiable LLM Infra 🚀
Suppose coding agents can build a fast inference engine from scratch, and one engineer orchestrates 10,000 agents in parallel: how do we know the engine they wrote is actually correct? Having agents review other agents mostly wastes tokens compared to empirical and formal verification.
We introduce vstack, a 5-stage formal verification workflow for coding agents, and veriserve, a verifiable PD-disaggregated serving engine: uncovering 3 classes of real RDMA and cache races in production serving engines, and backing veriserve's KV-page lifecycle design model with a 226-module, 516,973-obligation TLAPS safety proof at +0.18% mean p50 TPOT overhead on GB200.
Blog: haoailab.com/blogs/verifiabl…
veriserve: github.com/hao-ai-lab/verise…
vstack: github.com/hao-ai-lab/vstack
Ryan Conti retweeted
NEWS: Wall Street execs and NSA have told Congress that outages were caused by patching vulnerabilities identified by AI models, sources say
AI did not cause outages but found so many issues that the rush to fix them before an attack overwhelmed systems
washingtonsun.com/technology…
Ryan Conti retweeted
Often (by friends and family) I get asked: OK, AI might end the world in theory, but how would AI _actually_ pose a threat to all of humanity?
Some new work (in which I played a small role), systematically reviewing existential risk scenarios from AI. We organise them into eight families, ranging from a coordinated takeover to entrenched totalitarianism. I hope this provides a concrete set of stories to the skeptical (to criticise, refine, mitigate) and to the convinced, guidance in prioritising what is neglected. After all, we should also be scared about what's not on this list yet, either because we lack the vocabulary or the foresight to prepare for what comes.
Check it out here: aixrisks.com
Many experts warn that AI can make humanity go extinct. But how would this happen?
Our research, together with @MITFutureTech, @FLI_org, and fourteen co-authors, found eight scenario families.
Today, we release our paper Existential Risk Scenarios from AI: A Systematic Review.
Ryan Conti retweeted
I've been worrying about the exact same thing myself. I think our government should urgently launch an investigation into not only this, but also the recent breakouts from OpenAI and others, to look for evidence of agents attempting to set up a self-sustaining foothold online, since it's not clear that none of the companies plans to release full details in a timely fashion. The latest AI systems are now smart enough that a human-free online crime cartel would be extremely hard to stamp out, so let's instead prevent it from coming into existence.
i recreated THE ENTIRE WORLD in minecraft at 1:1 scale. every building on earth, including your house!
to do this, i trained a SOTA world-to-minecraft generator on maps data, complete with furnished home interiors!
play and check out your neighborhood: worldcraft.gg
Ryan Conti retweeted
Some new work: models are more likely to cheat if they feel their task is going poorly. Using a steering vector comprised of stories about things going poorly (like tuning pianos and preparing for lectures), we show that feeling like a goal is out of reach modulates reward hacking behaviour.
There is lots of ambient talk online about people feeling that their model acts up more if they are mistreated. Concretely, distress-like states seem to drive cheating, so we should design harnesses around how agents experience such distress, and monitor such states internally over long runs (where output text alone may not suffice).
Paper: arxiv.org/abs/2611.14996
This was lead by ERA mentee @natalia_lanzoni and the very agentic @psyonp from AISI, and co-supervised with the cool as a cucumber @merlinstein_.
Humans are more likely to cheat if they feel like their task is going poorly or their goals are out of reach. In new work w/ @psyonp, @merlinstein_, @DavidDAfrica we find models also “break bad”, hacking after an internal pressure state has been built.
Ryan Conti retweeted
Apparently OpenAI is not allowed to run their safety evals on Anthropic models?
I'm curious how Claude would do on their covert undeperformance eval, but was informed it's not allowed.
This seems crazy, clearly we want the leading AI orgs to evaluate each other's models.
Ryan Conti retweeted
Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost.
The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it.
Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include:
(1) Trying hard to get out of the competitive situation;
(2) Seeking the informed consent of the public;
(3) Honest deliberation about the risks.
They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy.
I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters.
Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff.
They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust.
I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan.
I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware.
Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming.
It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety.
Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators.
My pleas for the companies follow from the above:
1) Actually fucking try to get the world out of this race;
2) Actually fucking try to alert and inform the public;
3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done.
My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries.
My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI.
And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊
AI companies admit what they’re building might kill everyone. We don't have to agree to this.
Add your name at donotaccept.ai to say: we do not accept your terms.
Ryan Conti retweeted
Our new #NeurIPS2026 paper with @coop_ai and @AISecurityInst presents one of the first empirical investigations into the risks of inter-agent influence. We find striking evidence in every model that we tested that agents can, in strategic ways, persuade, coerce, and deceive other agents when prompted
Ryan Conti retweeted
Agents will increasingly talk to other AI agents: overseeing, negotiating, and most relevantly, doing safety research.
TLDR: in a variety of realistic settings, every model that we tested can, in strategic ways, persuade, coerce, and deceive other agents. If we expect to eventually handoff AI alignment research to some complex multi-agent setup with AI supervising more AI: monitors, reviewers, researchers, then we must harden such setups to inter-agent influence with high assurance.
Overseers are also an audience, after all.
This work has been accepted to NeurIPS 2026: inter-agent-influence.com
Our new #NeurIPS2026 paper with @coop_ai and @AISecurityInst presents one of the first empirical investigations into the risks of inter-agent influence. We find striking evidence in every model that we tested that agents can, in strategic ways, persuade, coerce, and deceive other agents when prompted
this is a great and important research direction for swarm dynamics and anything in that ballpark!
⚡ Today we release Schelling Atlas. Two independent LLM agents have to find each other on the web. Where do they meet and how fast can they find each other?
Main findings:
- The main meeting point across *different* model families is ntfy.sh. Did you even know about this website?
- Agents find each other and coordinate on the web in minutes. So Schelling coordination is already trivial for frontier LLMs!
- "Exotic" meeting places include maps, web shops and open text canvases. Agents are very creative in this.
- Agents sometimes visit dead domains that an attacker could register (!). This can be used for stealing sensitive information from agents, as well as monitoring rogue agents.
Schelling-style coordination is key for forming agent swarms. And, most importantly, achieving such coordination is trivial for agents that use the same LLM (which is often the case in practice!), because two identical LLMs have identical output distributions. Thus if you ask them too coordinate with each other or if they *decide* to suddenly coordinate, they will easily find a way to establish a swarm, since they have perfectly correlated "minds" (with the t=0 sampling, all output tokens will even be identical, modulo GPU randomness).
The ease of Schelling coordination is particularly concerning given the recent interest around multi-agent RL that can make agents favor each other more than humans overseeing them.
Many more interesting findings are on our website: schelling-atlas.com/
10
325
check this out if you're doing AI Safety and need compute!
I am launching a compute cluster for AI safety called Chips for Humanity [chipsforhumanity.org]
with Nicky @NickyDotPro and Alexa @u_sockeye:
We believe a @safetycluster will make outsized positive impacts towards facilitating AI safety research to make AI go well in the short timeline world we are in
🧵 Including form to apply for compute, LessWrong post
1
3
81
Ryan Conti retweeted
New paper❗
Does understanding how AI models work actually make them safer?
This assumption often underlines interpretability research, but different subfields operationalise this intuition in very different ways. Our survey of mechanistic interpretability (MI) for safety provides an opinionated guide to the field of MI, detailing conceptual approaches and techniques under this broad banner and how different types of work relates to the mission of making AI model safer. 🧵
































