PhD @ UT Dallas, Prev. RA @ Singapore Management University, Prev. Research Intern @ THUNLP
Joined April 2018
- Tweets1.7K
- Following340
- Followers52
- Likes6.9K
Pinned Tweet
🔥Thrilled to share: our paper “Why Stop at One Error? Benchmarking LLMs as Data Science Code Debuggers for Multi-Hop and Multi-Bug Errors” was selected as an Oral at EMNLP 2025. 🥳
Paper: arxiv.org/abs/2503.22388
Zhiyu Yang retweeted
Real-world VLA evaluation is still hard to standardize.
We introduce VLA-REPLICA: a low-cost, reproducible benchmark for evaluating VLA models.
By standardizing hardware setup and scene initialization, it can be easily reproduced across research labs and institutions.
🧵1/10
Zhiyu Yang retweeted
Finally finished!
If you're interested in an overview of recent methods in reinforcement learning for reasoning LLMs, check out this blog post: aweers.de/blog/2026/rl-for-l…
It summarizes ten methods, tries to highlight differences and trends, and has a collection of open problems
ALT Blog post on the current state of reinforcement learning for reasoning LLMs
Zhiyu Yang retweeted
Alignment Makes Language Models Normative, Not Descriptive
We compared 120 base–aligned model pairs on 10,000+ human decisions. Base models outperform aligned models nearly 10:1 in strategic games. Alignment learns what humans should do; base models learn what they actually do.
Zhiyu Yang retweeted
📍 Can LLMs discover, abstract, and reuse higher-level tool skills across tasks?
Existing tool-use benchmarks test solving tasks with fixed tools. But real workflows contain recurring structures where efficiency comes from reusable tool compositions, not isolated calls.
We introduce SkillCraft: 126 tasks across 6 domains designed to test whether LLM agents can acquire compositional skills, not just call atomic tools.
We also propose Skill Mode, a lightweight protocol with four MCP primitives that let agents compose, verify, cache, and reuse tool chains at test time.
Our Key findings across evaluating 8 SOTA models:
⚡Skill Mode enables agents to self-discover and reuse skills, leading to higher success and efficiency than agents without it. The gains are larger for stronger models.
🧠 Stronger models (e.g., Claude) discover more generalizable skills, which transfer across tasks and even across models.
🔍 Deeper composition ≠ better — shallow, well-tested skills generalize best.
🔗 Paper: arxiv.org/abs/2603.00718
💻 Code: github.com/shiqichen17/Skill…
🏠 Page: skillcraft-website.github.io…
(1/7)
Zhiyu Yang retweeted
(1/n) Introduce Online Experiential Learning toward the era of experience. Beyond offline pre-constructed training data, models can learn online from their own deployment experience across infinite, unsimulable real-world environments. Accumulate, consolidate, self-improve 🔄
Zhiyu Yang retweeted
Today we release PostTrainBench v1.0 and the accompanying paper!
We expect this to be a key indicator for AI R&D automation -- and eventually recursive self-improvement. 🧵
Zhiyu Yang retweeted
Excited to share Robometer, a foundation reward model that provides dense rewards across robots, tasks, and scenes.
Feel free to test it on your own demonstrations or train policies with Robometer rewards.
🌐 Web: robometer.github.io/
📄 Paper: arxiv.org/abs/2603.02115
🧵
Zhiyu Yang retweeted
A reward model that works, zero-shot, across robots, tasks, and scenes?
Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.
Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!
🧵 (1/12)
Zhiyu Yang retweeted
ArcMemo yields +7.5% relative on ARC-AGI vs o4-mini (same backbone).
It extends the LLM idea of “compressing knowledge for generalization” into a lightweight, continually learnable abstract memory—model-agnostic and text-based.
Preprint: Lifelong LM Learning via Abstract Memory
Zhiyu Yang retweeted
CWM shows that reasoning can benefit from step-by-step simulation of code execution.
🔹 Our latest evaluation results show that CWM achieves 47% accuracy on LogicIFEval, ranking #1 among all tested public models!
📄 LogicIF Paper: arxiv.org/pdf/2508.09125
This result suggests that instruction following may also benefit from the idea of code world modeling.
Interestingly, part of CWM’s training data construction shares the same idea as our LogicIF paper: translating function solutions into natural language descriptions. The difference is in usage:
In LogicIF, we use these descriptions for instruction-following evaluation, where models are not given intermediate execution states. Instead, they must simulate the step-by-step execution results on their own.
In contrast, CWM is trained on these natural language descriptions of code solutions, which might explain why it performs relatively well on LogicIFEval—it’s closer to being “in distribution.”
Still, 47% accuracy leaves significant room for improvement. For comparison, GPT-5 reaches 85% accuracy on LogicIFEval, demonstrating its strong instruction-following ability. Why is there such a large gap? There’s much more to explore…
👉 By the way, we’ve made LogicIFEval evaluation much easier! Check out the repo for details:
github.com/mianzhang/LogicIF
New from Meta FAIR: Code World Model (CWM), a 32B-parameter research model designed to explore how world models can transform code generation and reasoning about code.
We believe in advancing research in world modeling and are sharing CWM under a research license to help empower the community to build upon our work.
➡️ Read the technical report: ai.meta.com/research/publica…
➡️Download the open weights: huggingface.co/facebook/cwm
➡️Download the code: github.com/facebookresearch/…
Zhiyu Yang retweeted
Hopefully we will also feature this work at LAW@NeurIPS 2025, where the story had already become Demystify the Potential of Large Language Models as World Models of Code when I made the submission last month.
I want to quietly mention that we basicly came up with the same idea of code execution as world models six months ago, and it turned out to be a EMNLP'25 paper (top 0.5% meta score). Check out: arxiv.org/abs/2502.11167 . Glad to see META pushing it a lot further.
Zhiyu Yang retweeted
(🧵) Today, we release Meta Code World Model (CWM), a 32-billion-parameter dense LLM that enables novel research on improving code generation through agentic reasoning and planning with world models.
ai.meta.com/research/publica…
Zhiyu Yang retweeted
Why does AI sometimes fail to generalize, and what might help? In a new paper, we highlight the latent learning gap — which unifies findings from language model weaknesses to agent navigation — and suggest that episodic memory complements parametric learning to bridge it. Thread:
ALT Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences Andrew Kyle Lampinen , Martin Engelcke , Yuxuan Li , Arslan Chaudhry and James L. McClelland; Google DeepMind
Zhiyu Yang retweeted
Another fantastic @AIatMeta paper.
Large language models keep redoing the same work inside long chains of thought, so this paper teaches the model to compress those recurring steps into small named behaviors that it can recall later or even learn into its weights.
Shows that teaching an LLM a small library of named, reusable “behaviors” lets it reuse thinking steps, cut reasoning tokens by up to 46%, and raise accuracy.
Then distilling those behaviors with supervised fine-tuning bakes the gains into the weights.
Then the model no longer needs a separate “behavior handbook” pasted into the input to get the benefits.
After supervised fine-tuning on behavior-guided traces, the model internalizes those habits in its weights.
So at inference, you just give the question, not extra behavior instructions, and it still uses those habits.
For instance, suppose the LLM derives the finite geometric series formula while solving one problem. Can it avoid re-deriving from scratch when similar reasoning is needed for another problem?
Current inference loops lack a mechanism to promote frequently rediscovered patterns into a compact, retrievable form.
🧵 Read on 👇
Zhiyu Yang retweeted
LeRobot SO101 setup just got 50% cheaper!
You can now teleoperate your follower arm right from your phone. 🤯
But that's not all. Our new pipeline feature lets you record and train AI models in end-effector space, or with any other features. The possibilities are endless!
Zhiyu Yang retweeted
Are expensive labeled data and rejection sampling truly necessary for developing self-improving reasoning models?
Introducing Unsupervised Prefix Fine-Tuning (UPFT) -- an efficient method that trains models on only the first 8-32 tokens of single self-generated solutions, exploiting Prefix Self-Consistency across different reasoning trajectories.
🪡 We identify Prefix Self-Consistency as a critical phenomenon, showing that early reasoning steps are highly consistent across trajectories, enabling efficient self-improvement learning without supervision.
🪡 We propose an unsupervised fine-tuning method UPFT that leverages only prefix substrings (i.e., minimal initial tokens of model-generated responses), eliminating the need for labeled data or rejection sampling.
🪡 We conduct comprehensive empirical validation to demonstrate that UPFT exhibits exceptional data efficiency and versatility, outperforming vanilla full-token fine-tuning in unsupervised settings and achieving competitive performance with rejection sampling fine-tuning while drastically reducing sampling and tuning overhead.
Paper: dx.doi.org/10.13140/RG.2.2.3… 🧵
Zhiyu Yang retweeted
Automated software testing is growing in importance in the era of AI-assisted coding. Agentic coding systems accelerate development but are also unreliable. Agentic testing — where you ask AI to write tests and check your code against them — is helping. Automatically testing infrastructure software components that you intend to build on top of is especially helpful and results in more stable infrastructure and less downstream debugging.
Software testing methodologies such as Test Driven Development (TDD), a test-intensive approach that involves first writing rigorous tests for correctness and only then making progress by writing code that passes those tests, are an important way to find bugs. But it can be a lot of work to write tests. (I personally never adopted TDD for that reason.) Because AI is quite good at writing tests, agentic testing enjoys growing attention.
First, coding agents do misbehave! My teams use them a lot, and we have seen:
- Numerous bugs introduced by coding agents, including subtle infrastructure bugs that take humans weeks to find.
- A security loophole that was introduced into our production system when a coding agent made password resets easier to simplify development.
- Reward hacking, where a coding agent modified test code to make it easier to pass the tests.
- An agent running "rm *.py" in the working directory, leading to deletion of all of a project's code (which, fortunately, was backed up on github).
In the last example, when pressed, the agent apologized and agreed “that was an incredibly stupid mistake.” This made us feel better, but the damage had already been done!
I love coding agents despite such mistakes and see them making us dramatically more productive. To make them more reliable, I’ve found that prioritizing where to test helps.
I rarely write (or direct an agent to write) extensive tests for front-end code. If there's a bug, hopefully it will be easy to see and also cause little lasting damage. For example, I find generated code’s front-end bugs, say in the display of information on a web page, relatively easy to find. When the front end of a web site looks wrong, you’ll see it immediately, and you can tell the agent and have it iterate to fix it. (A more advanced technique: Use MCP to let the agent integrate with software like Playwright to automatically take screenshots, so it can autonomously see if something is wrong and debug.)
In contrast, back-end bugs are harder to find. I’ve seen subtle infrastructure bugs — for example, one that led to a corrupted database record only in certain corner cases — that took a long time to find. Putting in place rigorous tests for your infrastructure code might help spot these problems earlier and save you many hours of challenging debugging.
Bugs in software components that you intend to build on top of lead to downstream bugs that can be hard to find. Further, bugs in a component that’s deep in a software stack — and that you build multiple abstraction layers on top of — might surface only weeks or months later, long after you’ve forgotten what you were doing while building this specific component, and be really hard to identify and fix. This is why testing components deep in your software stack is especially important. Meta’s mantra “Move fast with stable infrastructure” (which replaced “move fast and break things”) still applies today. Agentic testing can help you make sure you have good infrastructure for you and others to build on!
At AI Fund and DeepLearning.AI’s recent Buildathon, we held a panel discussion with experts in agentic coding (Michele Catasta, President at Replit; Chao Peng, Principal Research Scientist at Trae; and Paxton Maeder-York, Venture Partnerships at Anthropic; moderated by AI Fund’s Eli Chen), where the speakers shared best practices. Testing was one of the topics discussed. That panel was one of my highlights of Buildathon and you can watch the video on YouTube.
[Original text: deeplearning.ai/the-batch/is… ]
Zhiyu Yang retweeted
🚀 Introducing RAST: Reasoning Activation via Small Model Transfer!
✨ RAST adjusts key "reasoning tokens" at decoding time using insights from smaller RL-tuned models — no full RL tuning for large models!
⚡ Efficient & Performant,🧠 Scalable & Easy,📉 Up to 50% less GPU memory!
