Postdoc @UCSanDiego, @mbzuai | working on world models | Previously @CityUHongKong | Visitor @UvA_Amsterdam
Joined November 2016
- Tweets77
- Following764
- Followers360
- Likes4.8K
Fan Feng retweeted
What should a robot remember? Can it learn to remember better than we tell it to?
Excited to share MEMI: Reverse-Engineering What Robots Should Remember
📈 97.0% end-to-end, 99.4% w/ low-level controller on RoboMME, nearly solving it for the first time
🧵👇
Fan Feng retweeted
🚨 Stop feeding your world model the same boring demos. Switch to free-range play data! 🚨
Our CoRL 2026 paper finds we can optimize autonomous data collection to focus on missing action consequences, achieving up to 85x the data scaling efficiency of expert data collection!🧵
Fan Feng retweeted
If video models scale like LLMs, they'll soon generate a perfect video plan for any embodied task. Do we need anything else? Introducing Mirror, which learns to follow these plans just by practicing, and even crafts an iron pickaxe in Minecraft w/o rewards or action labels! (1/n)
Fan Feng retweeted
Why learn everything from scratch when centuries of science have already taught us how the world works?
We’re excited to introduce ReRoute 🔀: bringing what-if reasoning to scientific AI models by combining observational data with partial mechanistic knowledge.
🔥 No controlled intervention data required.
🧵 1/7
Excited to share ReRoute, our new framework for scientific counterfactual prediction: informed by partial physics knowledge for much better generalization. 🌍
AI models can be remarkably accurate on familiar data, yet struggle with scientific ‘what-if’ questions
The key idea: Combine observational data with partial physical knowledge to predict unseen interventions—without requiring new controlled experiments.
Instead of retraining models on expensive new simulations, ReRoute uses known partial physical mechanisms to guide pretrained models through lightweight fine-tuning on existing data.
The results:
🌍 Up to 31.8% lower prediction error under severe CO₂ distribution shifts in climate emulators.
🔬 Up to 89.6% lower counterfactual error in controlled advection–diffusion experiments.
⚡ No new intervention data. No retraining from scratch. Just lightweight fine-tuning.
We also establish a causal identification result under explicit structural assumptions, with a machine-checked proof in Lean.
The bigger picture: Scientific AI doesn't always need more data or bigger models to generalize beyond what it has seen. Sometimes, knowing just a little about how the world works can make a big difference. @Dingling_Yao @kahaan_gandhi @FrancescoLocat8 @Caltech
📄 Paper: arxiv.org/abs/2610.02252
#AIforScience #ScientificML #ClimateAI #CausalInference #PhysicsAI
Fan Feng retweeted
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills.
The results:
• ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control.
• +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro.
• One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs.
The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning.
We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it!
Project + sessions: robot-agent-exploration.com/
Paper: assets.robot-agent-explorati…
GitHub: N
Project fully led by @RunhanH and @xczhangxx who are absolutely awesome.
How it works, experiments, and robot videos below 🧵
(1/N)
Fan Feng retweeted
Want to know how well agents can really control robots directly?
In RoboARCade, we systematically evaluate and diagnose frontier agents controlling robots directly using tens of thousands of dollars’ worth of agent trajectories. RoboARCade contains diverse, precise, and long-horizon tasks. Paper, code, assets and every trace are out today. roboarcade.org
Fan Feng retweeted
JEPAs can 𝐧𝐨𝐰 jointly imagine what happens and what to do next!
Meet 𝐋𝐞𝐖𝐀𝐌: A JEPA for the real world with
32.3× faster planning, 19 ms per plan
Improves success 28.6% → 89.7% on contact-rich manipulations
But also ... 🫶 a WAM for everyone with
17M · 1 GPU · 1 hyperparameter
Works on a real robot, not just a simulation !!
👉 le-wam.github.io/
Fan Feng retweeted
Is it possible to provably recover individual latent variables of the true world, even without reconstruction (e.g., JEPA)?
Yes, with DSReg, a simple regularization that can be applied post hoc to your pretrained model!
dsreg.github.io/
Fan Feng retweeted
Glad to announce that our paper has been accepted by #CoRL2026. See you in Austin!
Paper: arxiv.org/abs/2606.11525
Project: iwr-arxiv.github.io
Check out our recent paper with Fan Feng @ffeng01 and Caleb Chuck calcharles.github.io/.
We witnessed the success of unsupervised RL on locomotion, but found it hard to learn reusable skills on manipulation tasks. In this paper we dive deep into the mechanism of contrastive RL(CRL) and prove that successor feature(SF) has to memorize the piecewise nonlinearity when there is an interaction, compare to the smooth and linear representation in locomotion.
See the paper and project page for details.
Fan Feng retweeted
TC-WM is accepted at #NeurIPS2026! 🎉
A generic visual representation isn’t always a good latent state for planning and control.
For control, we need more than visual fidelity. Instead, we aim for task-centric representations that reflect physical dynamics and task-relevant information, while still leveraging the scalability of visual foundation models.
Compared with DINO-WM, TC-WM learns a lower-dimensional representation with higher latent-space utilization and diversity. These improvements translate into stronger performance, especially with the clearest gains on 3D contact-rich manipulations.
Webpage: minghaofu.com/tc-wm/
Paper: arxiv.org/abs/2605.25620
Code: github.com/MinghaoFu/TC-WM
Check out our new work on making contrastive RL work for interaction-rich manipulation, led by @TommyShen03, with Caleb (calcharles.github.io/). By learning the right geometry for representing manipulation skills, we train a robot air-hockey policy from scratch (w/o demos or task-specific rewards) and transfer it to the real world🧵
Can robots learn interactive manipulation skills from scratch without demonstrations or task-specific rewards?
Contrastive RL (CRL) offers a promising route: it excels in locomotion, yet remains brittle in long-horizon manipulation.
We introduce Interaction-Weighted Resampling (IWR), to address this failure mode for contact-rich, multi-stage manipulation. Across simulated tasks, it improves performance by 19.8% on average; on real-robot air hockey, success rises from 25% to 60%.
With @TommyShen03, Caleb Chuck and @ffeng01 across UC San Diego and UT Austin.
Technical blog: aetherlabs.ai/articles/the-g…
Paper: arxiv.org/abs/2606.11525
Project: iwr-arxiv.github.io/
🧵below
#RobotLearning #Robotics #Sim2Real #ContrasiveRL
This motivates Interaction-Weighted Resampling (IWR), which focuses contrastive learning on critical states around contact without changing the CRL objective by using directed sampling on critical interactive points
With this guided, interaction-centric sampling, IWR works across long-horizon, interaction-rich tasks, improving performance by 19.8% on average. It also transfers to real-world robot air hockey (Huge kudos to Caleb!): trained in simulation without demonstrations or task rewards, it raises success from 25% with standard CRL to 60%
Check out more results and details on our paper (arxiv.org/abs/2606.11525) and project page (iwr-arxiv.github.io/)
Fan Feng retweeted
I am honored to have received runner up best paper award at ICML's forecasting-workshop.github.… for my Bayesian forecasting paper. In my talk I showed some unpublished results on a more rigorous Bayesian approach. I have now added these results to the arxiv paper....
Fan Feng retweeted
The strongest healthcare LLM, custom-built for your enterprise, owned by you🌸
Meet @actAVAai Cura: 1T agentic model trained by recursive self-improvement for long clinical + health admin workflows
Try: actava.ai/cura
Share your use case: $20 credits + early access👇
Check out our recent work led by @Minghao__Fu, with @ncklashansen, on task-centric world models that learn task- and physical-aligned latents from pretrained embeddings like DINO and COSMOS (intermediate representations), while preserving visual fidelity and improving downstream control
1/
Most world models focus on generating high-fidelity futures: realistic videos, longer rollouts with expressive visual foundation models.
For control, we need more than visual fidelity. Instead, we aim for task-centric representations that reflect physical dynamics and task-relevant information, while still leveraging the scalability of visual foundation models.
Introducing TC-WM – a task-centric world model that learns grounded dynamics in a compact latent space while keeping the scalability of video foundation models.
- Technical blog: aetherlabs.ai/articles/task-…
- Paper: arxiv.org/abs/2605.25620
- Project: minghaofu.com/tc-wm
- Code: github.com/MinghaoFu/TC-WM
By Minghao Fu, Fan Feng, Nicklas Hansen, and Biwei Huang. (UC San Diego)
#WorldModels #RobotLearning #Robotics #EmbodiedAI #PhysicalAI #AI
Fan Feng retweeted
Heading to ICML 2026 next week!
I'll be presenting our work:
On Information Self-Locking in Reinforcement Learning for Active Reasoning of LLM Agents
📍 Poster
July 8 (Wed), 2:30–4:15 PM KST
Hall A #2014
The paper studies a failure mode of outcome-based RL for active reasoning, information self-locking, and introduces AREW, a simple advantage reweighting approach to improve information-seeking and belief-updating behaviors.
If you're attending ICML, feel free to stop by for a chat. I'd be happy to discuss active reasoning, RL, LLM agents, or related ideas.
See you in Seoul!
