Bill Shen retweeted
Anyone watching recursively self-improving agents knows how good they've gotten at optimising over a fixed benchmark with verifiable rewards, like math or coding. Our recent work, the Red Queen Gödel Machine (RQGM), co-evolves the evaluation itself together with the agent. 🧵
Excited to introduce our new paper, the Red Queen Gödel Machine (RQGM), that co-evolves evaluator with the self-improving agent.
Prior self-improving agents keep smashing fixed benchmarks - but those saturate faster than we can build them, and struggles for open-ended tasks.
We show RQGM delivers substantial gains:
- Coding: beats the prior SOTA while using 1.35-1.72× fewer tokens
- Paper writing: 1.78–1.86× higher acceptance
- Olympiad-level proof grading: 9% higher ground-truth accuracy
- Peer review: closing reviewer bias between AI and human work
32
Full paper: arxiv.org/abs/2606.26294
Co-authors: @Alex__Iacob, @itsmaddox_j, @DBBurkhardt, @meghdadkurmanji, Nurbek Tastan,
@lorenzosani97, @NickVenanzi, @AmbroiseOdonnat, @__cao_jamie__, Bill Marino, @xinchiqiu, @niclane7
@Cambridge_Uni, @nvidia, @flwrlabs, @mbzuai, @Inria
4
103
Thanks @imperial_nlp for having me as an external speaker - really enjoyed the talk and the inspiring discussions.
If you’re working on NLP / foundation models, I’d definitely recommend giving a talk here!
Replying to @imperial_nlp
We love hosting external speakers too, recent guests include @williamfshen (University of Cambridge) and @juand_r_nlp (the University of Texas at Austin). If you work in NLP / foundation models and would like to present your research, let us know and we'll make it happen!
📣 Our recent work at Meta: Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
Rubric-based judging and reward modeling are powerful for open-ended tasks, but scaling requires controlled generation, not just free-form lists from an LLM.
Big thanks to my amazing host and mentor @iliasl and co-authors: @xinchiqiu, Chenxi Whitehouse, @LisaAlazraki, @ShashwatGoel7, @fvancesco, @TimonWilli, @akhilmathurs
5
396
📊 Key results:
- Consistent gains across judges and datasets (e.g., up to +17.7 points on JudgeBench)
- Using RRD for reward modeling yields stronger learning signals (boosting reward by up to +160%), with gains transfer to downstream performance, including high-stakes domains
5
387
To address this, we propose RRD (Recursive Rubric Decomposition), a principled framework for controlled rubric generation that recursively decomposes coarse rubrics into discriminative sub-criteria, filters misaligned and redundant dimensions, and reweights correlated signals.
7
398
Rubrics help LLM judge capture nuanced, multi-dimensional human preferences and to turn open-ended quality into RL rewards. However, rubric generation is often messy: poor coverage, conflated dimensions, misaligned preferences, and correlated criteria, hurting their performance.
6
510
Excited to share our new work from MSL 🚀
We show that small agents can scale to long-horizon, complex tasks via coordination.
Strategy Auctions enable system-level gains: better task performance and cost–performance tradeoffs.
Capabilities emerge from how agents work together!
📣 New paper from my Meta Superintelligence Labs internship: Scaling Small Agents Through Strategy Auctions.
Small language models are cheap but don’t scale to long-horizon tasks. Strategy auctions help them punch above their weight. 🚀
(1/7) 🧵⬇️
2
163
Bill Shen retweeted
📣 New paper from my Meta Superintelligence Labs internship: Scaling Small Agents Through Strategy Auctions.
Small language models are cheap but don’t scale to long-horizon tasks. Strategy auctions help them punch above their weight. 🚀
(1/7) 🧵⬇️
Excited to share our work "LUNAR: LLM Unlearning via Neural Activation Redirection"
We propose a novel method that achieves SOTA unlearning effectiveness and significantly improves retained model's controllability.
Paper: arxiv.org/pdf/2502.07218
More in threads🧵
@xinchiqiu
Shout out to amazing collaborators: @xinchiqiu @meghdadkurmanji @Alex__Iacob @lorenzosani97 @yihong_thu @nicola_cancedda @niclane7
#CaMLSys, #UCL, #Meta, #LLM_unlearning
4
153
(4/N) "Experiments"
We show LUNAR delivers SOTA unlearning effectiveness and significantly improves retained model's controllability. It's also robust against various attacks and highly computationally efficient!
Visualization of results:
1
58
(3/N) "Method"
Grounded in the Linear Representation Hypothesis, LUNAR operates by redirecting unlearned representations to regions that trigger the model’s inherent ability to express its inability to answer.
2
46
(2/N): "Motivation"
Existing methods lack 𝙘𝙤𝙣𝙩𝙧𝙤𝙡𝙡𝙖𝙗𝙞𝙡𝙞𝙩𝙮, often leading to hallucinated, gibberish or monotonous responses. Worse, they require extensive LR tuning, be vulnerable to attacks, and still impose non-negligible training costs - hindering wide adoption.
2
49
Bill Shen retweeted
On Sunday at #KDD2024 (@kdd_news) our own Bill Shen (@williamfshen) presented PISTOL at the KDD workshop on Evaluation and Trustworthiness of Generative AI Models: genai-evaluation-kdd2024.git…. One of only a few papers selected for oral presentation 👏 arxiv.org/abs/2406.16810










