Nathaniel Li (@natliml) profile picture

evaluations @meta superintelligence labs 🇺🇸 🐻

Joined January 2023
Photo by Nathaniel Li (@natliml)Photo by Nathaniel Li (@natliml)Photo by Nathaniel Li (@natliml)Photo by Nathaniel Li (@natliml)Photo by Nathaniel Li (@natliml)
The Trump administration has delivered on a long list of conservative priorities, many of which seemed like pipe dreams for lonely academic discussion for decades. These are not small policy tweaks, they are major structural changes: shrinking and controlling the administrative state, getting immigration and mass migration under control, enforcing civil rights laws for everyone (read: also for white men, what a concept), defunding the institutional left’s woke NGO complex, turbocharging school choice and reining in universities legally and financially. They amount to an excellent groundwork for what must be recognized as a counterrevolution against the march through the institutions the left has been operating for a century.  But somehow, we’ve never been more deflated and blackpilled. This is madness! More: nationalreview.com/2026/11/t…
10
51
11
197
8,379
Muse Spark 1.3 is the first model we released in Muse, and we evaluate its capabilities and behaviors in those agentic environments. research.meta.ai/static/muse…
2
7
4
48
8,415
Nathaniel Li retweeted
Across America, incomes are rising & poverty is decreasing. 🇺🇸
2,955
1,527
208
6,207
312,525
Nathaniel Li retweeted
BREAKING: President Donald J. Trump just signed an Executive Order establishing a new federal academy, THE UNITED STATES SPACE ACADEMY. 🚀🇺🇸
2,157
5,883
1,190
33,682
7,845,996
We're releasing the preparedness report for Muse Spark Contemplating, MSL's extreme reasoning model, benchmarking its capabilities and behaviors in biology, cybersecurity, and more!
7
8
3
54
13,975
President Trump's AI Executive Order helps keep America in the lead on AI — encouraging innovation and taking security seriously. We appreciate the Administration's support for public-private sector collaboration and look forward to continuing to work with the White House as it implements the President's plan.
Today @POTUS signed an EO that keeps America leading in AI while putting frontier AI capabilities to work strengthening our cyber defenses. AI systems are now the most powerful tools we have ever had to harden our cyber infrastructure and stay ahead of adversaries. It is a real blessing that these capabilities are being developed by American industry, and not by those who would use them against us. whitehouse.gov/presidential-…
15
73
2
264
42,970
My team at MSL is hiring for talented ML researchers & engineers with experience in agentic security, dual-use capability evaluations, and/or post-training of frontier AI models! If this sounds exciting, please reach out and DM me with your resume!
2
15
1
329
39,262
Nathaniel Li retweeted
94
227
20
1,488
123,646
President Donald J. Trump launches the GENESIS MISSION - a national effort to accelerate scientific breakthroughs using AI and propel America into the GOLDEN AGE OF INNOVATION. 🚀
1,329
2,816
348
12,362
658,252
We're releasing a Preparedness report for Code World Model, assessing its bio, cyber, and chemical security capabilities in alignment with our Frontier AI Framework. We also provide nascent evaluations on the propensities of CWM, including honesty! ai.meta.com/research/publica…
We’re excited to share our preparedness report on Code World Model (CWM), FAIR’s latest open-weight model for code generation and reasoning. This report was developed by the SEAL team and the AI Security team, marking our first external publication since part of SEAL joined Meta just 1.5 months ago. It reflects our ongoing commitment to safety and alignment as we look ahead to future open-source and frontier models. We’re actively working to broaden our evaluation coverage and improve our processes, and we welcome collaboration and feedback from the broader research community. For more details, please read the full report: ai.meta.com/research/publica…
4
17
1,617
We're releasing a multimodal benchmark for troubleshooting complex virology protocols. Expert virologists score only 22.1%, even with internet access and answering questions in their subdomains of expertise. Frontier LLMs score up to 43.8%.
Can AI meaningfully help with bioweapons creation? On our new Virology Capabilities Test (VCT), frontier LLMs display the expert-level tacit knowledge needed to troubleshoot wet lab protocols. OpenAI’s o3 now outperforms 94% of expert virologists.
1
1
14
1,622
LLM benchmarks are getting quickly saturated. In response, we're releasing Humanity's Last Exam, a benchmark of extremely hard questions across math, sciences, humanities & more. Thanks to @cais, @scaleai, and thousands of expert question contributors.
We’re releasing Humanity’s Last Exam, a dataset with 3,000 questions developed with hundreds of subject matter experts to capture the human frontier of knowledge and reasoning. State-of-the-art AIs get <10% accuracy and are highly overconfident. @ai_risk @scaleai
1
20
1,966
ICYMI: New research from SEAL that demonstrates human red-teamers massively outperform automated methods over multiple turns. We also release MHJ, a dataset of multi-turn jailbreaks to help enable further research into multiturn red teaming
Who's better at LLM mischief — humans or AIs? Spoiler: It's us. Human red teamers achieve 70%+ attack success rates against LLM defenses that stump automated adversarial attacks. Why? We’re better at adversarial yapping.🧵
14
5
57
20,554
Nathaniel Li retweeted
Humans, noted virtuosi of adversarial yap, remain #1 at trolling LLMs! New research from @scale_AI's SEAL team shows human red teamers achieve 70%+ success rates against LLM defenses that stump automated attacks, exploiting their susceptibility to multi-turn jailbreaks.
Who's better at LLM mischief — humans or AIs? Spoiler: It's us. Human red teamers achieve 70%+ attack success rates against LLM defenses that stump automated adversarial attacks. Why? We’re better at adversarial yapping.🧵
1
7
55
15,339
Who's better at LLM mischief — humans or AIs? Spoiler: It's us. Human red teamers achieve 70%+ attack success rates against LLM defenses that stump automated adversarial attacks. Why? We’re better at adversarial yapping.🧵
9
19
12
95
66,534
We compile these results into Multi-Turn Human Jailbreaks (MHJ), a dataset of 537 multi-turn jailbreak conversations, with tactics and design considerations for every jailbreak. We publicly release MHJ to support research into more robust defenses. scale.com/research/mhj
1
12
753
MHJ was developed at @scale_AI ‘s SEAL team, with help from wonderful collaborators: @ziwen_h, Ian Steneker, Willow Primack, @goodside, @hughbzhang, @_zifan_wang, @CriMenghini, and @summeryue0! 📝static.scale.com/uploads/601…
2
8
759