Intelligence is born of bounded stochasticity.
California, San Diego
Joined December 2022
- Tweets141
- Following154
- Followers56
- Likes459
Pinned Tweet
Agents learn from what happened.
Can they learn what they should have seen coming?
Introducing 🔭 SRD: Self-Retrospection Distillation
- Prospective learning: align foresight with hindsight
- Learns from every rollout, even all-fail & groups
- Zero extra test-time inference cost
Haoxiang Zhang retweeted
What could you discover with AI agents on your team?
Join the Galileo Hackathon at UC San Diego HDSI
🧪 Track 1, Nov 4-18: design a scientific ML task
⚡ Track 2, Nov 21: solve fresh tasks with SIDERIUS
🏆 Awards in both tracks, GPU & API credits provided
Open to US universities & labs 👇
siderius.scitra.org/hackatho…
Introducing SIDERIUS, an open-source AI Research IDE powered by agents, for scientists.
Division of labor: you bring the science, agents do the ML engineering. Step in anytime: advise, take over any step, or run end to end.
siderius.scitra.org
#AIforScience #OpenSource
Haoxiang Zhang retweeted
Try it on play.honinglab.com/play/ and to play against one of our best models!
We gave open-source decision models two games. It did not go well. 🫠
🎮 Red Alert 2: the base model sold its own base. Then lost.
💥 Tank War: v1 spun in circles. v2 found the fire button… and shot itself.
So we let them keep learning: play → train → upgrade the harness → repeat. A few rounds later: model can evolve without human intervention
Then we put different models in one arena and let them fight it out. So far, no human has beaten the trained models. 😅
The lesson: a decision model isn't finished when it's released. Real environments keep throwing new situations at it, and a frozen model can't adapt. It needs continuous learning!
It's all open source: how we train these models to play, end to end. Next, we're turning it into training infra for any decision model.
⭐ Star it, fork it, or bring your own model to the arena:
GitHub: github.com/HectorHHZ/llm-hon…
RA2: github.com/HectorHHZ/llm-hon…
Tank War: github.com/HectorHHZ/llm-hon…
I also appreciate all the suggestions and feedback from: @b_vasilescu , Greg Ganger, @p0nk , @lateinteraction , @matei_zaharia , and @heathercmiller
And thanks to all the Contributors to the demo and the github repo!: @Jayzou3773, @brocoliman38951,
@ShangyinT , @Xuan04263449 , chenyang yang, tianyu chen , @AlexZhang152005, @Nickzz12138,@JasonWuzh, @Leow_sss730, @ChenMoneyQ , xingyuan ding, @JunchengLi , @LakshyAAAgrawal , @ao_qu18465
Very intuitive mind to save tokens.
⌘C ⌘V: Introducing Copy-Paste Language Models!
CPLMs learn to copy tokens better, improving downstream accuracy, CoT, long-context performance, and memory-efficiency for RAG at <3% overhead
PS: they (partially) mitigate the LM head bottleneck
Paper: arxiv.org/abs/2610.19247
Haoxiang Zhang retweeted
Recursive self-improvement is being hyped to the moon and back. I wanted to know how good frontier models actually are at a less-studied piece of it: curating pretraining data.
So I built PretrainGym at the @PrimeIntellect RL Residency 🧵
justin-chae.org/blog/pretrai…
Thank you @gurtej__gill_ !
Great paper from Salesforce, UCSD and Stanford.
It teaches AI agents to learn from their mistakes before they even start a task.
While doing Group Relative RLVR on hard agentic tasks, we sample a rollout group & every single trajectory fails.
And because there is zero reward contrast across the batch, the entire learning signal evaporates.
We throw away tons of compute, even though those failed runs are packed with useful clues about edge cases & environment traps the policy blindly stepped into.
The authors introduce Self Retrospection Distillation (SRD), and the intuition is super elegant.
Once an agent finishes an episode, the trajectory itself holds hindsight knowledge about what it actually needed to know to succeed.
Instead of just scoring the outcome, SRD distills that hindsight back into the model’s pre-interaction state thus essentially forcing the policy to anticipate pitfalls before it takes action.
What makes this clean from a deployment angle is that foresight is just an auxiliary loss during training, so you don't burn extra inference tokens or inference compute at test time.
The numbers in the sparse reward regime are super great: on a 2B model benchmark where standard RLVR hits an all-fail wall (literally 0.0% success), adding SRD jumps it straight to 60.6% under the exact same rollout budget.
This is a much smarter way to extract value from agent exploration. Its definitely worth a full read: arxiv.org/pdf/2610.08077
Fast pace!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Haoxiang Zhang retweeted
Are static benchmarks enough to evaluate VLMs in the real world? 🤔
In our #NeurIPS2026 paper, we move beyond traditional static benchmarks and introduce CEDI:
Give AI an oral exam !
We introduce CEDI, a framework that evaluates VLMs through dynamic, contextualized, multi-turn interactions. By grounding each conversation in a user’s goals, CEDI delivers contextualized evaluation tailored to real-world use cases.
We instantiate CEDI in visual hallucination, evaluate and reveal VLM's limitations that static benchmarks often overlook:
Key findings:
1. Contextualized evaluation reveals significantly more hallucinations than static benchmarks.
2. VLMs get lost in interaction: hallucinations accumulate and reinforce themselves over multi-turn interaction.
3. VLMs struggle with false premises and refusal scenarios.
📄 Full paper: arxiv.org/abs/2607.14499
💻 GitHub: github.com/williamium3000/ce…
🌐 Website: williamium3000.github.io/ced…
Joint work with @annazou1103 @IcyWang @ZiangXiao
See CEDI in action 👇
🎬 SRD in 2 minutes, hand-drawn style. Stay till the end ⭐
Design inspiration from huggingface.co/papers/2610.1… @llleizhang. Also check out their interesting paper!
Huge thanks to the amazing collaborators supporting this work:
Qinglin Chen, @hiroakiLhayashi, @zhuofengli96475, Siming Zhang,
@jxzhangjhu, @chenjx210734, @WUFang40615703, @lupantech,
@silviocinguetta, Juian McAuley @McAuleyLabUCSD, @jasonwu0731
📄 Paper: huggingface.co/papers/2610.0…
💻 Code: github.com/SalesforceAIResea…
Please don't hesitate to check them out =w=/
And if you find it interesting, a star 🌟 or upvote 💙 would be much appreciated!
8/
Where SRD sits.
On-policy post-training spans reward maximization (GRPO), mode seeking (OPSD), and mode covering (SDFT).
SRD builds on GRPO + OPSD and adds a new, prospective direction: not a new corner.
7/
When reward is silent.
Qwen3.5-2B on code, no dynamic sampling: 98% of groups fail every attempt.
GRPO ends at 0.0% training success.
Same rollout budget + SRD: 60.6%.
6/
Results on 10 benchmarks (math, code, search, ALFWorld, WebShop), Qwen3.5-4B/9B:
- improves 29/30 task categories
- 55/60 benchmark scores
- up to +24.2 pp (AIME26)
- helps GRPO, OPSD and RLSD alike
5/
Reward decides which lesson, not whether there is one.
✗ failures → Pitfall foresight
✓ successes → Knowledge foresight
An all-fail group: RL signal 0/8, SRD signal 8/8.
At test time no foresight is generated: same weights, same cost.
4/
How SRD works:
① Before acting, the policy writes a foresight: likely pitfalls, needed knowledge
② After acting, the same model sees the trajectory + answer and re-scores it
③ Token-level JSD pulls foresight toward hindsight, as a light auxiliary loss
3/
Our idea: prospective learning.
- RLVR: a trajectory becomes one scalar
- Self-distillation: hindsight is fed back to fix the actions
- Ours: hindsight goes further back, to supervise what the agent predicted before acting