Mattia Opper retweeted
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware.
It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning.
Read about the model architecture, distillation recipe, quantization and more: go.meta.me/4Bcvxol
Mattia Opper retweeted
Excited to see Muse Spark Open released! Three things I particularly appreciate: (1) efficiency before scaling, (2) opening up frontier architecture ideas, and (3) multimodal-native design from day one. A few personal thoughts 🧵
Mattia Opper retweeted
every time we think they've given up.
competes with glm 5.3 and kimi k3, which isn't super exciting at first glance, but it's small for that capability, and fully multimodal, and open
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
Excited to have been a part of this, MS open is really strong base and I think it can unlock a lot of cool research and applications! Personally I think the gated residuals are pretty cool too 😎
Mattia Opper retweeted
Meta has released Muse Spark Open, an 816B (42B active) open-weights model that scores 39 on the Artificial Analysis Intelligence Index, making it the #1 US open-weights model
Muse Spark Open (xhigh) scores 22 points above Muse Glimmer (high, 17), Meta's 30B dense open-weights model from August. It is also ahead of NVIDIA's Nemotron 3.5 Super VL (26), the next highest-scoring US open-weights model, which launched today
@AIatMeta shared access with us ahead of public release for benchmarking. Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release!
Key Takeaways:
➤ Open weights at the level of Meta's August flagship. Muse Spark Open scores within a point of Muse Spark 1.2 (xhigh, 40). It is stronger on AutomationBench-AA and has a lower hallucination rate, but weaker on GDPval-AA v2.1, CritPt and AA-Omniscience accuracy. Within Meta's current lineup, it sits 13 points behind Muse Wonder (max, 52) and 17 points behind Muse Wonder Full (max, 56)
➤ Token-efficient, at a moderate cost per task. Muse Spark Open uses 40k output tokens per Intelligence Index task, the fewest among the nine highest-scoring open-weights models: e.g. 16% fewer than Muse Spark 1.2 (xhigh, 48k) and 43% fewer than GLM-5.3 (max, 71k), its closest Chinese peer by size. Together AI, Fireworks and Baseten offer it at launch for $1.25/$4.25 per 1M input/output tokens ($0.15 cached input). At that pricing it costs $0.84 per Intelligence Index task. That is less than Muse Spark 1.2 ($0.97) at the same per-token prices and less than half of GLM-5.3 (max, $2.01), but about three times DeepSeek V4.1 Flash (max, $0.27)
➤ A relative weakness in agentic work compared to Chinese open-weights leaders. Among the nine highest-scoring open-weights models, Muse Spark Open ranks last on the four agentic evaluations in the index but fourth on the other six combined, and it leads all open-weights models on AA-Omniscience (25). GLM-5.3 (max), the closest Chinese model by size, owes all of its 6-point lead to the agentic evaluations
➤ Self-hostable on a single node, or two Mac Studios. BF16 weights are ~1.6TB. Meta's NVFP4 checkpoint quantizes only the 256 experts, cutting the weights to ~530GB. Meta says this runs on four Blackwell Ultra GPUs or eight H200s, or on two 512GB M5 Ultra Mac Studios using MLX. The architecture uses 62 layers with grouped-query attention (16 KV heads), 8 of 256 experts active per token, and a 1.8B-parameter Perception Encoder for vision
Other model details:
➤ Parameters: 816B total, 42B active (MoE)
➤ Context window: 1M tokens
➤ License: Muse Model License, a custom license that requires "Built with Muse Spark Open" attribution and a separate agreement with Meta for Model-as-a-Service businesses with over $20M in annual revenue
➤ Input modalities: text, image, video and audio. Output: text
➤ Availability: weights on Hugging Face (BF16 and NVFP4), with API access via Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter and Together AI
Mattia Opper retweeted
Chelsea’s keynote at COLM highlighted how natural language feedback offers much richer supervision than scalar rewards.
This feels especially important for continual learning: a computer-use agent may not get a clean scalar reward to learn from, but it does see how the environment changes and what the user says next.
With ECHO, we explored the simplest way of learning from these signals on the fly by training the model to predict the environment’s response. Learning to predict user feedback is a natural extension. What other objectives could let agents learn directly from these interactions without needing a scalar reward?
Mattia Opper retweeted
Looking for research engineer contractors for our team, working on AI Research Agents (AIRA). We are building agents that can perform diverse research tasks, run their own experiments, share knowledge and self-improve. Apply here: uk.meta.talentnet.community/…
Mattia Opper retweeted
🚨🧵 New paper!
Mech interp has developed many ways to probe and intervene on model internals. It's natural to ask: is there a unifying picture?
In this work, we derive 4 popular interp methods from a single representational hypothesis: Tensor Product Representations (TPRs)
1/n
ALT Overview slide of "A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability". The title is on the top, followed by authors: Zhang Enyan and Tom McCoy Left: An elephant viewed from different perspectives, where each viewer think the elephant feels like a fan, snake, wall, and rope, respectively. This classic story is compared with a black-box LLM, where it is viewed as probes, SAEs, causal traces, and analogies. Center: explanation text "We unify a variety of interpretability methods under Tensor Product Representations (TPRs), and show that they can be constructed directly from TPRs with comparable performance ", followed by a schematic of how a Tensor Product Representation (TPR) is constructed. The schematic has arrows linking to four cells, each containing an interpretability method, demonstrating they can be viewed as applications of TPR operations
Mattia Opper retweeted
Visual test I promised: Muse Glimmer 30B finding checkboxes on a synthetic healthcare form.
Image in, coordinates out fully local on my Mac Studio. It returned 10 boxes and matched all 10 states.
What's your experience with OCR and vision in Glimmer 30B model?
Mattia Opper retweeted
First impressions on Muse Glimmer!
It's incredibly fast for a dense model, currently running an average of 208tps with a max of 274tps on a single 5090 with their DFLASH config.
Comparatively, though, both using Open Code, Qwopus Coder (with thinking off) produced a much better shark survival game than the one I got from Glimmer. Meta's new dense model is currently just lacking some HTML canvas taste, but this is something that can be added via SFT as long as the model is stable and capable from a back-end programming perspective.
And it seems to be, without a doubt. The big kicker here is that I ran this at extra high thinking, and it did not take long at all to run. Our current local leader, Qwen 27B 3.6, has a tendency to overthink, but with glimmer, that is not the case.
Right now, my recommendation for general local programming (Apps, Games, Websites, Visual Tools) in this class is still Qwopus Coder with thinking disabled, or Qwopus Fusion with thinking enabled. Of course Shark Survival is a very basic domain-specific test, but I find that the result scales very well across many domains. If we're going to be shipping apps generated entirely locally, visual taste is somewhat of a bare minimum requirement, solely in my opinion, and Qwen's models in this class offer significantly more at the moment.
That's actually why I initially started getting into finetuning with Qwen 3.5, they were the first base that was able to do really good front-end with some opus-trace fine-tuning. Qwen 3.6 has taste even in the base model, and we know Qwen 3.8 is going to blow us all away!
Regardless, this looks like a very tempting new base model. As a first offering from Meta in this class for a long time, I am incredibly impressed and elated to have it. We now finally have a proper Single GPU frontier race, instead of us just begging Qwen for more releases.
Single GPU open frontier model race is a VERY good thing.
Please keep pushing @meta
Muse glimmer is out! I hope it will be really fun and useful model for people to customize, research and experiment with - congrats to the whole team it was a blast creating this together with you all 🥳
Introducing Muse Glimmer, an open-weight 30B-parameter model optimized for local, always-on agent workflows.
Muse Glimmer delivers strong performance on key agentic use cases and benchmarks compared with leading models in its size category, and is designed to run entirely on consumer hardware like a Mac or PCs with performant GPUs.
In keeping with our long tradition of sharing fundamental AI research, we’re releasing model weights under a permissive Apache 2.0 license.
🧵👇
Mattia Opper retweeted
Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models.
Mattia Opper retweeted
1/ big announcement today: we will be releasing an open weight version of muse spark 1.2 soon.
we also are releasing muse glimmer, a 30B agentic model with open weights under apache 2.0. muse glimmer can run on 24GB of VRAM without losing agentic reliability. 🧵
Mattia Opper retweeted
I think this is a very nice deal for random hobby stuff.
In my experience, the model is an absolute beast in terms of execution.
It's not quite as clever/clean as Sol/Fable in complex situations yet, but it easily holds up to Luna/Terra/Opus while being super fast.
(At least in my internal use, not sure about public use yet)
Mattia Opper retweeted
Transformers use the same hidden state for two competing roles: maintaining state and predicting the next token. 🔀
Splitting these roles into two separate streams outperforms standard Transformers.
Introducing the State-Prediction Separation (SPS) Transformer.
🧵
Mattia Opper retweeted
Excited to share that I will be presenting my 1st year PhD work at #ICLR2026 @iclr_conf next week in Rio 🇧🇷
❓In-Context Learning (ICL) has been a widely used paradigm to adapt LLMs for a task with a few examples or shots. But does it also work for multimodal examples? 🧵
Mattia Opper retweeted
Super proud of how this work turned out!
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
Mattia Opper retweeted
@karpathy 's autoresearch lets an AI experiment autonomously overnight, but the AI (e.g., Claude Code) itself remains fixed. It gets better results, but it doesn’t improve at improving itself. Hyperagents change that. They evolve the process of improvement itself.
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
Mattia Opper retweeted
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).