
evaluations @meta superintelligence labs 🇺🇸 🐻
Joined January 2023
- Tweets28
- Following861
- Followers953
- Likes313
Nathaniel Li retweeted
The Trump administration has delivered on a long list of conservative priorities, many of which seemed like pipe dreams for lonely academic discussion for decades.
These are not small policy tweaks, they are major structural changes: shrinking and controlling the administrative state, getting immigration and mass migration under control, enforcing civil rights laws for everyone (read: also for white men, what a concept), defunding the institutional left’s woke NGO complex, turbocharging school choice and reining in universities legally and financially.
They amount to an excellent groundwork for what must be recognized as a counterrevolution against the march through the institutions the left has been operating for a century.
But somehow, we’ve never been more deflated and blackpilled. This is madness!
More: nationalreview.com/2026/11/t…
Muse Spark 1.3 is the first model we released in Muse, and we evaluate its capabilities and behaviors in those agentic environments.
research.meta.ai/static/muse…
Nathaniel Li retweeted
BREAKING: President Donald J. Trump just signed an Executive Order establishing a new federal academy, THE UNITED STATES SPACE ACADEMY. 🚀🇺🇸
Nathaniel Li retweeted
President Trump's AI Executive Order helps keep America in the lead on AI — encouraging innovation and taking security seriously.
We appreciate the Administration's support for public-private sector collaboration and look forward to continuing to work with the White House as it implements the President's plan.
Today @POTUS signed an EO that keeps America leading in AI while putting frontier AI capabilities to work strengthening our cyber defenses.
AI systems are now the most powerful tools we have ever had to harden our cyber infrastructure and stay ahead of adversaries. It is a real blessing that these capabilities are being developed by American industry, and not by those who would use them against us.
whitehouse.gov/presidential-…
Nathaniel Li retweeted
President Donald J. Trump launches the GENESIS MISSION - a national effort to accelerate scientific breakthroughs using AI and propel America into the GOLDEN AGE OF INNOVATION. 🚀
We're releasing a Preparedness report for Code World Model, assessing its bio, cyber, and chemical security capabilities in alignment with our Frontier AI Framework. We also provide nascent evaluations on the propensities of CWM, including honesty!
ai.meta.com/research/publica…
We’re excited to share our preparedness report on Code World Model (CWM), FAIR’s latest open-weight model for code generation and reasoning.
This report was developed by the SEAL team and the AI Security team, marking our first external publication since part of SEAL joined Meta just 1.5 months ago. It reflects our ongoing commitment to safety and alignment as we look ahead to future open-source and frontier models. We’re actively working to broaden our evaluation coverage and improve our processes, and we welcome collaboration and feedback from the broader research community. For more details, please read the full report: ai.meta.com/research/publica…
We're releasing a multimodal benchmark for troubleshooting complex virology protocols. Expert virologists score only 22.1%, even with internet access and answering questions in their subdomains of expertise. Frontier LLMs score up to 43.8%.
LLM benchmarks are getting quickly saturated. In response, we're releasing Humanity's Last Exam, a benchmark of extremely hard questions across math, sciences, humanities & more. Thanks to @cais, @scaleai, and thousands of expert question contributors.
Nathaniel Li retweeted
ICYMI: New research from SEAL that demonstrates human red-teamers massively outperform automated methods over multiple turns.
We also release MHJ, a dataset of multi-turn jailbreaks to help enable further research into multiturn red teaming
Nathaniel Li retweeted
Humans, noted virtuosi of adversarial yap, remain #1 at trolling LLMs!
New research from @scale_AI's SEAL team shows human red teamers achieve 70%+ success rates against LLM defenses that stump automated attacks, exploiting their susceptibility to multi-turn jailbreaks.
Who's better at LLM mischief — humans or AIs? Spoiler: It's us.
Human red teamers achieve 70%+ attack success rates against LLM defenses that stump automated adversarial attacks. Why? We’re better at adversarial yapping.🧵
We compile these results into Multi-Turn Human Jailbreaks (MHJ), a dataset of 537 multi-turn jailbreak conversations, with tactics and design considerations for every jailbreak. We publicly release MHJ to support research into more robust defenses.
scale.com/research/mhj
1
12
753
MHJ was developed at @scale_AI ‘s SEAL team, with help from wonderful collaborators: @ziwen_h, Ian Steneker, Willow Primack, @goodside, @hughbzhang, @_zifan_wang, @CriMenghini, and @summeryue0!
📝static.scale.com/uploads/601…
2
8
759
























