@lightetal

I’m a PhD researcher @MSFTResearch @Caltech @RPI, prev @NECLabsAmerica @AsariAILabs working on LLM-agents, reasoning, RL, test-time scaling, and code generation

 San Francisco, CA
jonathanmli.github.io
Joined June 2023
  • Tweets330
  • Following889
  • Followers921
58Media
We trained an AI agent on its own explanations, not its code. It got better at coding. No teacher. No verifier. No RL updates. No extra context at test time. Here's how. Links below. 🧵
8
10
8
103
12,247
My coding agents are running. Figured I should too
1
2
20
749
What I read at #COLM2026: the 5 papers I liked most, ranked by community likes: 1. IdeaScientist: agents trained with RL to generate grounded research ideas. @Jiarui_Liu_ nitter.cf/Jiarui_Liu_/status/210… 2. LLM-as-a-Verifier: a weaker model checking a stronger one’s work. @Azaliamirh nitter.cf/Azaliamirh/status/2108… 3. CORAL: autonomous multi-agent evolution for open-ended discovery. @ao_qu18465 nitter.cf/ao_qu18465/status/2108… 4. Actor-Curator poster. @lightetal nitter.cf/lightetal/status/21082… 5. GitSwarm: inference that builds on itself across long tasks. @veds_12 nitter.cf/veds_12/status/2108605… My overall impression is that verification is now the bottleneck for both agents and reasoning. Which COLM paper did I miss?
Jiarui Liu @COLM
@Jiarui_Liu_Oct 8
Excited to share IdeaScientist, our research from my internship at Meta! 🚀 Can AI agents learn to generate novel, grounded scientific ideas? We introduce IdeaScientist, a framework that trains specialized agents to identify research gaps, discover useful connections across scientific domains, and turn them into concrete research proposals. 🔍 Key findings: • +14% overall performance over open autoresearch baselines • 25% improvement in novelty, showing that explicit training can improve scientific ideation • 75–92% human preference win rates against six open baselines in blind evaluations • Cross-domain retrieval substantially improves the transfer of ideas between research fields We also introduce Svalbard Idea Vault, a collection of 2.77M decomposed research ideas for training and evaluating scientific ideation. 📄 Paper: arxiv.org/abs/2610.04074
3
6
1
44
6,064
Our Actor-Curator poster at #COLM26 in case you missed it yesterday :)
Jonathan @COLM
@lightetalFeb 26
Post-training LLMs is like mixing a cocktail: Too much easy data → no learning Too much hard data → instability Wrong balance → collapse And today, we mix it by hand. What if the data mixture could be learned instead of hand-tuned? arxiv.org/abs/2602.20532 🧵👇
1
3
40
1,341
Jonathan @COLM retweeted
Yao@COLM@zhengyao_guOct 7
At #COLM2026? Care about RL and Curriculum Learning for Agents? Do bandit research? Come chat with us about Actor–Curator this afternoon! 🕟 Today, Oct 7, 4:30–6:30 PM PDT 📍 Poster #100, Franciscan C Hilton San Francisco Union Square
Yisong Yue
@yisongyueFeb 26
Actor-Curator does self-curriculum design for post-training, by dynamically choosing task difficulty instead of hand-mixing data. Yields up to 80% faster convergence compared to vanilla post-training. Awesome work by @zhengyao_gu @lightetal @ihazelye @zou_henry43378
5
2
8
270
Shockingly Simple Self-retrospection Improves Agentic Models Without RL "Can a language-model agent improve its future actions by training only on explanations of its own experience? We investigate this question by studying Retrospection-Only Fine-Tuning (ROFT), a minimal online procedure designed to isolate the effect of explanation-only training on subsequent behavior. The agent attempts a task, observes available feedback, generates a retrospective explanation, and is fine-tuned with a next-token prediction loss on the explanation tokens alone." link: arxiv.org/abs/2609.35741
11
42
4
308
17,514
At #COLM2026? Come chat about Actor–Curator this afternoon! Happy to talk curriculum learning, bandits, and RL post-training, or just say hi 👋 🕟 Today, Oct 7, 4:30–6:30 PM PDT 📍 Poster #100, Franciscan C Hilton San Francisco Union Square
Jonathan @COLM
@lightetalFeb 26
Post-training LLMs is like mixing a cocktail: Too much easy data → no learning Too much hard data → instability Wrong balance → collapse And today, we mix it by hand. What if the data mixture could be learned instead of hand-tuned? arxiv.org/abs/2602.20532 🧵👇
4
18
830
Jonathan @COLM
@lightetalSep 29
We trained an AI agent on its own explanations, not its code. It got better at coding. No teacher. No verifier. No RL updates. No extra context at test time. Here's how. Links below. 🧵
2
7
543
I’ll be at @COLM_conf in San Francisco this week presenting our latest work on curriculum learning for RL post-training! #COLM26 Feel free to hit me up if you want to chat about agents, post-training, agentic coding, and self-improvement!
2
2
35
1,233
My dms are open :)
2
144
Can a coding agent learn to use fewer tokens and turns, without a length penalty? We asked it to reflect on how to reach the solution faster, then trained it on those reflections. 11.7% fewer tokens. 13.1% fewer turns. Compared with training on general reflections.
3
15
755
Retrospections can help steer model behavior without explicit reward shaping. For details, see arxiv.org/abs/2609.35741

Shockingly Simple Self-retrospection Improves Agentic Models Without RL

People learn not only by repeating successful actions, but also by recounting and explaining their experiences, revising their understanding to guide future behavior. Can a language-model agent...

arxiv.org
2
134
Really great work from @emilianopp_ !
Emiliano Penaloza
@emilianopp_Sep 30
Want to train your SWE agents 2× faster 🏎️💨? We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer. Unlike other works that assume a fixed compaction strategy, we show that KV-streams can work with any compaction method in agentic tasks. Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀 arxiv.org/pdf/2609.35750 🧵
4
556
Jonathan @COLM retweeted
scottjmaddox
@scottjmaddoxSep 30
Unexpectedly effective alternative to RL?: provide feedback, let the model contemplate the feedback in chain of though, and then fine-tune with next token prediction on the contemplation tokens. Credit assignment flows back through the full rollout to penalize poorly chosen actions. Unlike RL, it appears to work even for tasks where all the original rollouts failed. This could be a game changer.
Tanishq Mathew Abraham, Ph.D.
@iScienceLuvrSep 29
Shockingly Simple Self-retrospection Improves Agentic Models Without RL "Can a language-model agent improve its future actions by training only on explanations of its own experience? We investigate this question by studying Retrospection-Only Fine-Tuning (ROFT), a minimal online procedure designed to isolate the effect of explanation-only training on subsequent behavior. The agent attempts a task, observes available feedback, generates a retrospective explanation, and is fine-tuned with a next-token prediction loss on the explanation tokens alone." link: arxiv.org/abs/2609.35741
1
1
7
349
Jonathan @COLM retweeted
Zhening Li (Zed)
@zli11010Sep 30
strongly agree with the approach of a minimal harness. if it’s a maximally expressive code-mode agent loop then you can also do some really long-horizon tasks without a memory system and self-improvement without a meta-harness :D arxiv.org/abs/2609.26891

Harness as a Language: A Minimalist Agent Framework With Maximal...

Modern language-model agents are built around the agent loop: the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between...

arxiv.org
Michele Catasta
@pirrohSep 29

Free the models: harness design at the frontier

Article · @pirroh
1
5
469
What if I told you an agent could start solving a task after failing all initial attempts? On two hard coding tasks, training on its own retrospections did exactly that. No successful example to imitate. A failed attempt can still contain a useful lesson.
Jonathan @COLM
@lightetalSep 29
We trained an AI agent on its own explanations, not its code. It got better at coding. No teacher. No verifier. No RL updates. No extra context at test time. Here's how. Links below. 🧵
2
17
1,158
An all-failure group gives binary-reward GRPO no relative reward signal. But its attempts can still expose mistakes worth explaining. Zero observed successes doesn’t mean zero useful experience.
103
5/6 This isn’t ROFT vs. RL. Add retrospection training to GRPO and learn from both rewards and explanations. The combination achieved the highest SWE-bench Verified score in our comparison. Learning to explain complements learning to do.
1
2
12
519
6/6 Paper & acknowledgments Huge thanks to my coauthors: @ccui9 @beanie0__0 @creus_roger @emilianopp_ @Zhengyan_Shi @murefil @Cote_Marc @ericxyuan @kim__minseon And thank you @MSFTResearch for supporting this work! Paper: arxiv.org/abs/2609.35741

Shockingly Simple Self-retrospection Improves Agentic Models Without RL

People learn not only by repeating successful actions, but also by recounting and explaining their experiences, revising their understanding to guide future behavior. Can a language-model agent...

arxiv.org
1
2
16
578
Load more