Swarnadeep Saha retweeted
Upgraded the models for Muse and Meta Al, bringing Active Memory and Active Focus from Muse Wonder to everyone. In Muse, these new specialized models are 2.4x less likely to forget critical info and are 67% better at self correcting and completing everyday tasks which means less repeating yourself and fewer dead ends.
More on the models: research.meta.ai/blog/specia…
Swarnadeep Saha retweeted
Claim: Humans & AI are better together if we want to recursively (co-)improve to a brighter future for *humanity*
Instead of racing towards methods for AI replacing us, work on how to make AI work *with* us!
New paper that makes 4 contributions here:
1) The role of humans in directing research agents: Autobenchmark
2) Reward hacking in autoresearch agents: HackBench
3) Training agents to interact with humans: AutoInteract
4) Training agents to communicate with humans: unslopping AI
Full Paper: arxiv.org/abs/2610.38882
Blog posts for each section:
1) facebookresearch.github.io/R…
2) facebookresearch.github.io/R…
3) facebookresearch.github.io/R…
4) facebookresearch.github.io/R…
Swarnadeep Saha retweeted
Introducing *HackBench* 🤡💀⏱️📊 ⏲️ 💪🔧 🔥
- Detecting & localizing reward hacking in autoresearch agents
- Frontier models fail
- Dataset on HuggingFace
- Recipe to collect natural & synthetic hacks
Blog post: facebookresearch.github.io/R…
🧵1/5
Swarnadeep Saha retweeted
Muse Wonder is out, with strong performance. My team contributed to memory (& MemoryBench, listed), legal, financial, and various other agentic tasks @shehzaadzd @tesatory Jack Lanchantin @swarnaNLP & others!
Swarnadeep Saha retweeted
Muse Wonder is SOTA on legal reasoning.
–We want to help people understand their rights and make better decisions and close the legal literacy gap. Legal literacy gap is one of the main causes of the higher Gini index of a society. We would love to level the playing field and empower everyone to make the best decisions in their situation.
–Legal has been an awesome testbed for in-context learning. Laws are different in different countries / states and are ever changing. Instead of training the model to memorize all the laws and cases, we train the model to effectively retrieve relevant laws and cases and leverage them to learn in-context to provide most accurate and up-to-date results. Legal documents can be long and cumbersome, which give us the opportunity to evaluate and improve our long-context reasoning capabilities.
h/t @swarnaNLP @jaseweston, Tiara, Vivian, and all the amazing collaborators.
research.meta.ai/blog/muse-w…
It’s been an incredible few months working with the team building Muse Wonder. Among other things, it brings frontier capabilities in personal superintelligence spanning many domains like legal, finance, and health.
Hope you like it!
Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.
1
10
644
Swarnadeep Saha retweeted
Claim: teaching AIs to be better collaborators with humans improves the quality of their work!
🤖🤝Introducing *AutoInteract* 🔁💬🧑💻🧑🤝🧑✨🌱
- creates train data to improve AI's collaborative abilities
- simulates the way humans act & constructs challenging multi-turn interaction tasks
- Improves LiveCodeBench-Pro, Multi-turn LCB-Pro & SWE-together
We expect the recipe generalizes to much more!
Blog post: facebookresearch.github.io/R…
🧵1/5
Swarnadeep Saha retweeted
Earlier this year, our focus shifted from Olympiad-level problems to tackling open research questions. Having majored in mathematics, I realized that I lacked the natural genius required to solve the specific open problems I was passionate about. That realization led me to launch my career in machine learning. Since last year, however, my objective has evolved: rather than trying to solve these open problems entirely on my own, I am now dedicated to training AI models to solve them. For example, in collaboration with a few graduate school friends who are now math professors, we have already made meaningful progress on an open problem in arithmetic physics.
Industry-wide progress along this trajectory has been nothing short of phenomenal. Mathematics resembles a vastly expanded game of Go—it functions within a structured environment and possesses intrinsic self-verification. We experienced mathematics' "AlphaGo moment" this year, marking a point of no return. We anticipate that a similar intelligence transition will unfold across other scientific disciplines, albeit at a different pace owing to physical verification constraints in biology (to be verified in the human body), chemistry, and physics (to be verified in the universe). As this intelligence revolution unfolds, our primary mission is to empower everyone to navigate and thrive through this smooth transition.
The models used to tackle these open research questions were built on earlier iterations of Muse Spark; our current and upcoming models are already significantly more capable. We will keep you posted.
research.meta.ai/blog/solvin…
Swarnadeep Saha retweeted
Following gold-medal-level performance from our AI models across five competitions in mathematics, physics, and chemistry, we asked a harder question: can AI contribute when a problem is genuinely open and without an existing solution path?
Over the past several months, mathematicians worked with Muse Spark 1.1 and Muse Spark 1.2 in Thinking Mode through the regular meta.ai chat interface, with no custom research scaffold, to find solutions to such problems.
Our goal wasn't to mass-produce papers, but empower researchers. Every collaboration followed the same principles: mathematicians guided the research, a second group of mathematicians then reviewed the work, each paper marks which passages were drafted primarily by humans or AI, and each credits the prior research it builds on. Where other teams independently announced solutions to the same problems, we acknowledge their work as well.
Today, we're sharing six papers from that collaboration.
🧵👇
Great work by Seungone where he carefully studies the impact of humans in building hard benchmarks!
Yes, agents do most of the work but humans play a critical role too.
⏱️⚙️Introducing *AutoBenchmark* 📊🏁
- Creating benchmarks automatically
- Benchmarking benchmark creation
- Studying the role & impact of humans in the loop
Blog post: facebookresearch.github.io/R…
Key takeaways:
1) We find human-agent collaboration gives big wins over agents alone
- fine-grained feedback in ideation stage crucial
- autobench can be used to measure this in the future with stronger agents
2) We show that it's possible to make *autoresearch benchmarks* for AI research using this recipe – full recursive improvement loop!
3) Important Ingredients: Autobenchmark creation works best with feedback from two sources: benchmark solvers + external verifiers (human+AI).
🧵1/5
My favorite bits:
1. By *meta-optimizing* rubrics, we learn *features* that make expert-human writing superior & thus RL against those rubrics can produce *superhuman* text too.
2. In an online setting, this co-improves both rubrics+model, so extends beyond this specific task.
Claim: we've solved the AI slop problem (!) 💩🧹✨
Blog post: facebookresearch.github.io/R…
🧵1/5
Key idea: take *expert* human writing and learn rubrics that find the gap between experts and models. Train with those rubrics.
We train with RL-XAR (RL with eXpert Aligned Rubrics) & see large performance gains on writing scientific paper sections, Pulitzer prize novel continuations and high quality Wikipedia pages.
Another expert-authored professional reasoning benchmark where Muse Spark has held the top spot for quite some time now.
Leaderboard spotlight!
With @Muse on top of the app store, a special shoutout to @AIatMeta for Muse Spark 1.3. The @ScaleAILabs team worked closely with them on evaluations, training data, and testing.
Muse does well on tool use, multi-turn conversation, tutoring, professional reasoning, and end-to-end software engineering. Different tasks, but each asks a model to hold context across long horizon workflows, make judgments under uncertainty, and recover from partial failures.
PRBench is where that's clearest. Its tasks are written by domain experts and scored on rubrics that reward professional utility, not trivia-style correctness. Muse has held the top spot in finance and law since 1.1, and 1.3 extends the lead.
10
1,046
Closer to the frontier than ever!
Try our Muse Spark 1.3 at developer.meta.com/ai/models…
Meta has released Muse Spark 1.3, their fourth Muse Spark model release in five months. Muse Spark 1.3 (max), which is in limited preview for Meta’s partners, scores 62 on the Artificial Analysis Intelligence Index, behind only Claude Fable 5.1 and Claude Opus 5. The variant available now, Muse Spark 1.3 (xhigh), scores 61 and ties with GPT-5.6 Sol (max) and Grok 4.6 (high). Both variants’ gains come primarily from improvements in agentic work and scientific capabilities
Muse Spark 1.3 (xhigh) enters the Artificial Analysis Intelligence Index at 61, up 4 points from Muse Spark 1.2 (57, August) and 8 points from Muse Spark 1.1 (53, July). It enters tied with GPT-5.6 Sol (max), Grok 4.6 (high), and Claude Opus 5 (high), and behind Claude Fable 5.1 (max, 66), Claude Opus 5 (max, 63), and Claude Fable 5 (max, 62)
Muse Spark 1.3 (max), which is in a limited preview stage, lands at 62. This higher index score is enabled by gains vs. Muse Spark 1.3 (xhigh) in Tau3-Bench Banking (52% vs. 47%) and GDPval-AA v2 (1,754 Elo vs. 1,709). Muse Spark 1.3 (max) is second only to Claude’s Fable and Opus variants in total score
Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release!
Key Takeaways:
➤ Continued improvement on agentic knowledge work tasks. At the launch of Muse Spark 1.2, we noted its significant gains in agentic knowledge work performance vs. Muse Spark 1.1. The latest iteration continues this trend, with Muse Spark 1.3 (xhigh) demonstrating a notable 12-point gain vs. Muse Spark 1.2 in Tau3-Bench Banking (35% to 47%), a 5-point gain in Terminal-Bench 2.1 (80% to 85%), and a new GDPval-AA v2 Elo of 1709 against its predecessor’s 1615. Muse Spark 1.3 (max) improves further on Tau3-Bench Banking (52%) and GDPval-AA v2 (1,754 Elo). This Tau3-Bench Banking score is #1 among all models. Muse Spark 1.3 (max) achieves these higher agentic work scores by using more turns and total reasoning tokens, reasoning 62% more on GDPval-AA v2 and 28% more on Tau3-Bench Banking compared to Muse Spark 1.3 (xhigh)
➤ The lowest cost per task for any model at 59+ on the Artificial Analysis Intelligence Index. Muse Spark 1.3 (xhigh) costs $0.55 per Intelligence Index task at Meta's unchanged $1.25/$4.25 per 1M token pricing ($0.15 for cached input), with its peers GPT-5.6 Sol (max) and Grok 4.6 (high) costing $0.95 and $0.94 respectively, a 70%+ premium. This places Muse Spark 1.3 (xhigh) on the Pareto frontier for Intelligence vs. Cost per Task. Its cost per task is higher than Muse Spark 1.2 ($0.40 per task), driven by ~57% more input tokens per task on agentic evaluations, with output tokens up only ~8%. Pricing for Muse Spark 1.3 (max) is not yet publicly available
➤ Scientific Reasoning results rose across the board, led by CritPt. CritPt was the standout non-agentic score gain vs. Muse Spark 1.2, with a material +8 points for the xhigh variant (18% to 26%), and GPQA Diamond achieved +4 points (90% to 94%), while Humanity’s Last Exam and SciCode each gained a more modest 2-3 points (45% to 47% and 56% to 59%, respectively). Muse Spark 1.3 (max) achieved roughly similar scores to the xhigh variant, gaining 2 points in Humanity’s Last Exam, tying on GPQA Diamond, and losing a point on CritPt vs. Muse Spark 1.3 (xhigh)
➤ Minor regressions in only two evaluations. Both Muse Spark 1.3 (xhigh) and Muse Spark 1.3 (max) dropped 4 points in AA-LCR (83% to 79%) when compared to Muse Spark 1.2, and AA-Omniscience (Accuracy) fell 3 points for xhigh and 1 point for max. The drops in AA-Omniscience (Accuracy) are due to a higher abstention rate (not answering questions when unsure), which also lowered the hallucination rate for Muse Spark 1.3 (xhigh)
Other model details (xhigh variant):
➤ Context window: 1M tokens, unchanged from Muse Spark 1.2
➤ Pricing: unchanged from Muse Spark 1.2: $1.25/$4.25 per 1M input/output tokens, with cache hits discounted to $0.15 per 1M
➤ Input modalities: text, image, video
➤ Availability: Meta's first-party API and Muse Code
1
29
739
Swarnadeep Saha retweeted
Muse Spark 1.3 is rolling out today with frontier performance almost too cheap to meter. This is the biggest jump we've made so far on coding and agentic work. Try it in Muse Code and our API.
Next up 🍉 and Muse Spark open weights releases coming soon.
Swarnadeep Saha retweeted
Muse Voice Transcribe is MSL's first real-time audio perception model -- rolling out today. SOTA in streaming speech-to-text, it handles speaker diarization, and endpointing natively in a single model.
OG of open-weight models is now back with Muse Glimmer. Enjoy!
Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models.
10
748
Muse Spark 1.2 is also the leading model on Harvey's Legal Agent Bench, improving further on its predecessor!
It's generally a good model for different professional tasks, so give it a shot!
Muse Spark 1.2 is the first model to crack 60% on Finance Agent v2, our benchmark that gives models the job of a financial analyst.
At $0.77/test it is 6.7x cheaper than the previous #1, Opus 5 ($5.12), at twice the speed.
Swarnadeep Saha retweeted
To understand whether we're making genuine progress on reasoning, we entered our AI models in five STEM Olympiad competitions.
The results:
🏅 Asian Physics Olympiad (APhO): Perfect score, theory exam
🏅 International Physics Olympiad (IPhO): Perfect score, theory exam
🥇 International Mathematical Olympiad (IMO): Gold medal
🥇 International Chemistry Olympiad (IChO): Gold-medal-level performance
🥇 Romanian Masters of Mathematics (RMM): Gold-medal-level performance
The types of problems in the Olympiad competitions are exceptionally hard, demanding deep chains of reasoning, creative insight, and flawless argumentation. To test pure reasoning capability, we disallowed all tool use, meaning no search, no coding, and no calculator.
We have deep admiration for the contestants and committees behind these competitions, and are grateful for their support in enabling our participation.























