Noah Provenzano

🚀 I am excited to share our recent work, which I will present at #COLM2026 🚨 Our data curation framework improves LLMs’ long-context reasoning capability in open domains, even via self-distillation! We introduce Pi^2, a pipeline that 1) transforms Wikipedia tables into high-quality long-context QA data and 2) back-translates reasoning traces with realistic context. 📜: openreview.net/forum?id=4yDp… 💻: github.com/vt-pi-squared/pi-… 🧵:👇 (1/n)

hosting the Self-Improving Agents social at @COLM_conf tomorrow - 4 talks: @sethkarten @yoonholeee @qinyuan_ye and Noah Provenzano - panel: @HuaxiuYaoML @ZhiruoW @jasonwu0731 my 4th one after ICLR, ICML and NeurIPS, and I still love hosting these. 1-2:30pm, Franciscan D. come by if you're at COLM luma.com/m8v4fq76

🌟New paper alert: The Master Key Hypothesis🌟 Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead? We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections. Takeaways: 1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key. 2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces 3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger 4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer. 5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.

🌟New paper alert: The Master Key Hypothesis🌟 Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead? We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections. Takeaways: 1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key. 2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces 3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger 4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer. 5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.

Introducing EvoSkill: a framework that analyzes agent failures and automatically builds the missing skills, leading to rapid improvement on difficult benchmarks and generalizable skills across use-cases. +12.1% on SealQA +7.3% on OfficeQA (SOTA) +5.3% on BrowseComp via zero-shot transfer from SealQA Read more below 🧵