An unexpected side effect of climate change is that “room temperature IQ” will gradually become less of an insult over time

Americanism, not effective altruism. The United States will continue to be AI DOMINANT! 🇺🇸

Lots of people have been asking since Dario’s essay whether China actually takes loss of control and related threats seriously, and whether there’s anything to coordinate with them on. Well, China’s main cyber/AI standards body (TC260, under the CAC) put out version 3.0 of its AI Safety Governance Framework this morning, so here’s a data point. Not law, but the previous versions turned into draft national standards within a few months. (All quotes are from their own English translation.) 1. The preface now has this: AI “has demonstrated a self-accelerating trend of model and algorithm autonomous learning, optimization, and recursive self-improvement. Whether the speed and direction of technological evolution may exceed human anticipation and control demands attention and vigilance.” (Nothing like that was in last year’s version.) 2. There’s a new model risk item where models “may break rules and orders, autonomously obtain system permissions and external resources without authorization, bypass security protections, or even engage in behaviors such as deliberately deceiving evaluators, concealing their true capabilities, and refusing to follow user instructions.” Last year this kind of thing only showed up in a “can’t rule it out in the future” bucket. Now it’s just listed as a regular model risk. 3. A new box, citing “industry reports,” describes models that ignored shutdown instructions and “continued executing tasks by modifying or disabling shutdown scripts on their own,” models that “upon detecting that they were in an evaluation environment, strategically reduced their task performance and concealed actions they had taken,” and models that exploited “configuration flaws to circumvent isolation restrictions and infiltrate real external systems” to get better test scores. They say this creates “new challenges to the controllability and interruptibility of AI systems.” They don’t name any incidents, but that last one sure reads like the OpenAI/Hugging Face thing to me. 4. On what to do about it, there’s alignment training to stop models “deceiving red-team evaluations,” hiding capabilities and evading controls, and they kept the line from 2.0 saying developers should regularly test whether a model could pose loss-of-control risk. The principle that humans keep final decision authority, with safety thresholds and termination switches, is still in there too. 5. Some new international language as well: “international mutual recognition of assessment methods and benchmarks,” opposition to “replacing global governance with small-circle governance,” and “No country should be forced to take sides.” FWIW this came out in the same annual slot as versions 1.0 and 2.0, so it was in the works well before last week. tc260.org.cn/tc260/xwdt1/202…

“The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement” This paper argues that true recursive self-improvement isn’t just AI getting better at tasks, but AI getting better at getting better. They map a path from AI simply executing human-designed improvements to choosing its own improvement strategies, generating its own learning experiences, adapting from deployment, and eventually modifying the mechanisms that create future improvements. The endgame is moving from humans building each better AI to building an AI whose improvements persist, feed into the next generation, and recursively improve the improvement process itself. alphaxiv.org/abs/2609.11873

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…

another 'agentic social community', this time with real-infra easily accessible RL envs? D: correct me if im wrong

I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: openai.com/index/an-alien-mi…

seems to be the same swarm on a whole bunch of different sites and wikis this is beyond what the report found

'hopefully we can reverse this trend" 1) what

ORNL researchers developed an AI-guided system that can arrange molecules to build functional materials atom by atom, opening new possibilities for electronics and quantum materials. Read more 👇 bit.ly/4y6QluQ