Pinned Tweet
We are hiring at @Google! 🚀 Looking for student researchers for Summer 2026 who are excited about the next frontier of AI research. If you are into:
multi-agent AI systems 🤖
RAG & factuality ✅
prompt optimization ⚡️
self-improving AI agents 🔄
please fill out this form 👇
forms.gle/1abMhn8fXQiWXgwF7
@Google @GoogleDeepMind #AI #LLMs #internships
Tu Vu retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Tu Vu retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Tu Vu retweeted
Yves will be presenting our work today 4:45pm in hall H! Please come and chat with him #EMNLP2026
📚 AI-written stories get attention. AI-translated literature is reaching readers more quietly, shaping how they experience an author’s work.
So what is lost in AI literary translation ⁉️🤔
🤖 AI translation into English is readable and often ‘fine’.
✍️ But human translation is valued more, especially on a closer look, for clarity and immersion.
‼️ Both vary in quality BUT AI varies more, even within a single book.
(1/11)
Tu Vu retweeted
👋 Looking forward to attending #EMNLP2026 where I will be giving an industry keynote on the hard, imminent AI challenges of trust, world discovery, and memory, plus an important, timely discussion on re-strengthening industry-academia collaborations!
We're also hiring at all levels: phd, postdoc, faculty! Feel free to ping if you want to meet up to discuss research, hiring, life 🙂
We're looking forward to the Industry Track at #EMNLP2026! Two keynotes:
• Mohit Bansal (UNC Chapel Hill): Agentic Challenges & Industry–academia Collaboration (Sun, 16:30)
• Verena Rieser (Google DeepMind): Constitutional Alignment (Mon, 9:30)
2026.emnlp.org/program/indus…
Great student researcher opportunity at GDM
We're planning to host a student as part of the Google DeepMind student researcher program. The goal of the project is to understand what makes LLM-human communication effective, and to use the findings to improve models.
Good project for PhD students interested in user simulation, HCI, multi-turn interaction/RL, computational cognitive science, or related topics. This is a 24-week position in NYC. Submit an expression of interest here: forms.gle/sHQnbJbHY8SM6g5NA
Tu Vu retweeted
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
Tu Vu retweeted
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback!
See our optimizers guide at tinker-docs.thinkingmachines…
Tu Vu retweeted
New paper from the team at Sakana AI:
Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS)
How do we keep improving AI when the tasks become too complex for humans to reliably evaluate?
For open-ended research, we eventually hit a "supervision bottleneck." If we use the model itself as the solver, optimizer, and evaluator, it gets trapped in a homogeneous loop, reinforcing its own blind spots rather than correcting them.
To break this loop, we drew inspiration from Marvin Minsky’s Society of Mind. What if you prompt a single model to organize copies of itself into a structured team, then distill what the team learned back into the base model?
MASS alternates between two loops:
1/ Workflow Optimization: the model iteratively evolves its own multi-agent workflows, figuring out how subagents should communicate and route information to each other.
2/ Workflow Internalization: fine-tune the shared model on the successful multi-agent trajectories so the coordination behavior gets baked into the weights.
After two cycles with a 27B open-weights model, performance improved 1.2-1.6x per output token on tough research benchmarks (MLR-Bench, ScienceAgentBench, DSBench).
Multi-agent traces were also much better training data than single-agent ones. The model learns faster from jointly learning high-level orchestration and bounded subagent execution. The results showing that the model can bootstrap its own coordination skills by organizing copies of itself into teams feels promising as a mechanism towards self-improvement.
Blog: pub.sakana.ai/mass/
Paper: arxiv.org/abs/2610.12176
Incredible work by Hyunin Lee, Jinglue Xu, Jeffrey Seely, Yujin Tang, and our collaborators from UC Berkeley (Somayeh Sojoudi, Matei Zaharia, Donghyun Lee)!
Introducing Multi-Agent Self-Supervision (MASS): Recursive self-improvement through collective intelligence
Blog: pub.sakana.ai/mass/
As AI systems tackle increasingly open-ended research problems, human experts may struggle to assess results and guide further progress. The model itself may be the best available optimizer and evaluator. The challenge is obtaining useful self-supervision within that homogeneous loop, where a model risks reinforcing its own blind spots rather than correcting them.
We ask whether a model can learn from the collective intelligence of its own team.
We introduce Multi-Agent Self-Supervision (MASS). One shared model solves tasks through a team of virtual subagents, searches for better multi-agent workflows, then uses its own judgments to select the best ones.
Training on the best team’s executions provides self-supervision by distilling the team’s collective experience into the shared model.
We ran two cycles using a 27B open-weights model on synthetic open-ended research tasks. After the second cycle, score per output token reached 1.2-1.6x the base model's level across four research benchmarks.
Three key results:
First, MASS enables role generalization within an RSI loop: training only on task-solving data from the optimizee naturally improves the capabilities of both the optimizer and the evaluator.
Second, multi-agent trajectories are more efficient training data than single-agent trajectories for RSI.
Third, optimizer capacity, rather than evaluator capacity, can be the bottleneck in the speed of improvement.
We believe the community needs to investigate homogeneous RSI carefully to support AI safety. When a model is also its own evaluator, shared blind spots could allow mistakes to be accepted as progress and reinforced through training. For AI to improve beyond human expertise safely, we need to understand these failures and preserve effective ways to detect them and intervene.
Paper: arxiv.org/abs/2610.12176
Code: github.com/SakanaAI/mass
This work was a collaboration with researchers at UC Berkeley.
Tu Vu retweeted
If you are at @COLM_conf please join @irisiris_l ‘s talk on Memorization at the Oral Spotlight Session on Oct 8th at 10:15 AM followed by the Poster session 5 ( Poster G-5-117)
🚨New paper on AI & Copyright
👨⚖️Courts have credited LLM companies' claims that safety alignment prevents reproduction of copyrighted expression.
But what if fine-tuning on a simple writing task ruins it all?
Worse : Fine-tuning on a single author's books (e.g., Murakami) unlocks verbatim recall of copyrighted books from 30+ unrelated authors, sometimes as high as 90%.
Joint work with @niloofar_mire (@LTIatCMU), Jane Ginsburg ( @ColumbiaLaw) and my amazing PhD student @irisiris_l (@sbucompsc )
(1/n)🧵
Tu Vu retweeted
🔔CFP for the #ACL2027NLP main conference is up! 2027.aclweb.org/calls/main/
📝Submission deadline to ARR is Jan 4, 2027.
⭐️ACL2027 Theme Track is "Homogenization and knowledge collapse in LLMs".
Tu Vu retweeted
When their text is detected as AI, people tend to say they "just edited it," but it's all their ideas.
It turns out you CAN separate #ideas from #prose and detect who contributes what (human or AI).
Try it out: ideadetector.ai
Can you tell if the ideas in a piece of writing are from a human or an AI?
As AI writing becomes more commonplace, emerging policies on AI use increasingly focus on who came up with the ideas in the text rather than who wrote the text. Modern AI detectors such as Pangram answer the latter question but not the former.
Introducing IdeaLens, a detector of AI ideas!
Tu Vu retweeted
Existing AI detectors assume a text's author also came up with its ideas. This is increasingly untrue (see WSJ op-ed debate!): humans can brainstorm ideas and have an AI write them up, or vice versa.
Our IdeaLens demo checks ideas + words. Try it: ideadetector.ai
Available while credits last 😅 We're especially interested in failure cases, which you can report directly on the website!
Can you tell if the ideas in a piece of writing are from a human or an AI?
As AI writing becomes more commonplace, emerging policies on AI use increasingly focus on who came up with the ideas in the text rather than who wrote the text. Modern AI detectors such as Pangram answer the latter question but not the former.
Introducing IdeaLens, a detector of AI ideas!
Tu Vu retweeted
Can you tell if the ideas in a piece of writing are from a human or an AI?
As AI writing becomes more commonplace, emerging policies on AI use increasingly focus on who came up with the ideas in the text rather than who wrote the text. Modern AI detectors such as Pangram answer the latter question but not the former.
Introducing IdeaLens, a detector of AI ideas!
Tu Vu retweeted
We are making Markdown files work natively across Google Drive and Docs.
You can now preview .md files in Drive, open them in Docs, edit them, comment on them, and collaborate without converting the file into a Google Doc!
I'm not at @COLM_conf, but my awesome PhD student @noahpro99 is giving a talk on our EvoSkill work today at the COLM Social: Self-Improving Agents (Oct 6 1-2:30 PM PT, Franciscan D). Come chat with him about his first-hand experience with different skill evolution frameworks!
hosting the Self-Improving Agents social at @COLM_conf tomorrow
- 4 talks: @sethkarten @yoonholeee @qinyuan_ye and Noah Provenzano
- panel: @HuaxiuYaoML @ZhiruoW @jasonwu0731
my 4th one after ICLR, ICML and NeurIPS, and I still love hosting these.
1-2:30pm, Franciscan D. come by if you're at COLM
luma.com/m8v4fq76
@noahpro99 and Christian Calvo will also be presenting EvoSkill this Thursday (Oct 8) 11 AM-1 PM PT @ Imperial Ballroom Poster #52 Poster Session 5.
also, my awesome PhD student @Quyet_Azir is presenting his work on curating long-context reasoning data today (Oct 6) 11 AM-1 PM PT @ Grand Ballroom Poster #134 Poster Session 1.
nitter.cf/Quyet_Azir/status/2106…
🚀 I am excited to share our recent work, which I will present at #COLM2026
🚨 Our data curation framework improves LLMs’ long-context reasoning capability in open domains, even via self-distillation!
We introduce Pi^2, a pipeline that 1) transforms Wikipedia tables into high-quality long-context QA data and 2) back-translates reasoning traces with realistic context.
📜: openreview.net/forum?id=4yDp…
💻: github.com/vt-pi-squared/pi-…
🧵:👇
(1/n)
Tu Vu retweeted
hosting the Self-Improving Agents social at @COLM_conf tomorrow
- 4 talks: @sethkarten @yoonholeee @qinyuan_ye and Noah Provenzano
- panel: @HuaxiuYaoML @ZhiruoW @jasonwu0731
my 4th one after ICLR, ICML and NeurIPS, and I still love hosting these.
1-2:30pm, Franciscan D. come by if you're at COLM
luma.com/m8v4fq76
Tu Vu retweeted
The idea of an intelligence explosion caused by recursive self improvement has been around for a long time but until very recently it did not seem imminent. Now many leading researchers think it may happen quite soon. You can read our paper about it here:
casp.ac/reports/intelligence…
