phd student @carnegiemellon • interning @amazon far • prev @nvidia, @georgiatech • reinforcement learning, differentiable simulation, robotics
Joined July 2018
- Tweets64
- Following379
- Followers469
- Likes1.6K
heterogeneous parallel sim for robotics in newton physics! diversity scaling
Today we’re open-sourcing Steel, our internal physics solver built with @NVIDIA.
It runs simulations up to 13× faster than PhysX while using ~3× less GPU memory.
Steel unlocks scaleable simulation for manipulation, for everyone.
2
168
gradients from differentiable world models for fast policy learning! amazing work (and visual style) from @qiayuanliao
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/
7
546
Eliot Xing retweeted
When SceniX joined World Labs, we said spatial intelligence was never only about perceiving and generating virtual and physical worlds, but also interacting with them. Today, we’re sharing early results from that vision: building worlds that train robots. 🌎🤖↓
Eliot Xing retweeted
RIO, a new framework from CMU, provides a unified interface for robot control, data collection, teleoperation and AI deployment!
It gives researchers a lightweight, modular foundation for deploying robots quickly on any platform.
ri.cmu.edu/robot-io/
we built a flexible infra layer for physical ai and robotics research, check out @PAOKApproves's post! switch between different tabletop arms (single, bimanual) & humanoids, for policy inference, teleop, or data collection. all in python, easy for humans and agents to build on.
Physical AI is more than a model. Missing reusable infrastructure is often the painful part.
At #RSS2026, meet RIO: Flexible Real-Time Robot I/O for Cross-Embodiment Robot Learning (1/8)
autoresearch hill-climbing with a fleet of real-world robots 🤖🤖🤖🤖🤖
Autoresearch just left the sandbox and entered the embodied world.
We are excited to introduce 𝐄𝐍𝐏𝐈𝐑𝐄: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop:
reset the environment → search the literature → implement ideas and build the infra → train and deploy → self-verify → analyze the logs and rewrite the code → repeat, until the policy is reliable in the real world. No human in the loop.
Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties.
We envision the bottleneck in robotics shifting — from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own.
🔗 research.nvidia.com/labs/gea…
From @NVIDIA @CMU_Robotics @Berkeley_AI
🧵
1
6
1,116
Eliot Xing retweeted
Newton 1.0 is now generally available. 🙌
Take robot learning to the next level with:
🤖 Stable Articulated & Complex Mechanism Simulation – accurate, reliable machine modeling.
🖐️ High-Fidelity Hydroelastic Contact Modeling – realistic soft contact and touch-based interactions.
🧵 Deformable Body Simulation – simulate cables, cloth, rubber, and other elastic materials with VBD.
⚡ Accelerated Robot Learning at Scale – seamless integration with open simulation and learning frameworks, NVIDIA Isaac Sim and Isaac Lab for scalable workflows.
Learn how to integrate this open-source physics engine into your workflow: nvda.ws/3NGTzUo
#NVIDIAGTC
check out the latest from @TairanHe99 and co. on visual sim2real humanoid loco-manipulation!! really enjoyed the timeline documenting the sim2real journey
The Journey.
This wasn't an overnight success. It took 6 months of building visual sim, distributed training, and infra from scratch.
Robotics is hard. We’ve documented our failures and our path to success here: viral-humanoid.github.io/#si…
7/
amazing work from @ChaoyiPan on physics-based retargeting for dexterous robot hands and humanoids that runs out of box on multiple physics simulators!
🕸️ Introducing SPIDER — Scalable Physics-Informed Dexterous Retargeting!
A dynamically feasible, cross-embodiment retargeting framework for BOTH humanoids 🤖 and dexterous hands ✋.
From human motion → sim → real robots, at scale.
🔗 Website: jc-bao.github.io/spider-proj…
🧵 1/n
2
6
786
Eliot Xing retweeted
How do you give a humanoid the general motion capability? Not just single motions, but all motion?
Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control.
We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames.
But tracking alone is not enough; we show how to make a useful control system out of it:
- Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game.
- VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset.
- VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically.
- VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution!
We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain.
This is the generalist "System 1" for humanoids. 🚀
Project: nvlabs.github.io/SONIC/
#Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
Eliot Xing retweeted
What if robots could improve themselves by learning from their own failures in the real-world?
Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks.
PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel.
Quick 🧵
Eliot Xing retweeted
The laws of physics apply everywhere. ⚛️
Co-developed by NVIDIA, @GoogleDeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the @linuxfoundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning.
Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 nvda.ws/4gP2woL
#CoRL2025
cool work from @Jsphamigo! decouple first-order RL with data from nondifferentiable simulation + gradients from a learned differentiable dynamics model, for large sample efficiency gains and quadruped sim2real
Introducing our new work DMO: Decoupled Model-based policy Optimization! First-order gradient RL that unrolls trajectories with high-fidelity sims & computes gradients via learned models.
Paper & demos: machines-in-motion.github.io…
#CoRL2025 w/ @Rk4342R
3
1,091
Eliot Xing retweeted
📣MuJoCo announcement 📣
Thrilled to share that @GoogleDeepMind has unveiled MuJoCo-Warp at @nvidia's #GTC25!
🚀 We've expanded our open-source MuJoCo simulator with MuJoCo-Warp, leveraging NVIDIA’s Warp framework for incredible acceleration. This marks a significant step in making high-performance simulation more accessible.
Eliot Xing retweeted
🚀 We've already extended MOE to various dexterity projects including dynamic pen spinning (arxiv.org/abs/2411.12734) and learning in-hand manipulation skills from demonstration (arxiv.org/abs/2503.01078). 🧵6/7
Safety is hard to ensure in human-robot interaction with rigid manipulators. We present a soft robot system to provide safer interaction for hair care.
@UksangYoo has also already used his soft manipulator for more dexterous tasks like pen spinning, check out his full thread!
🎉Excited to share that our paper was a finalist for best paper at #HRI2025! We introduce MOE-Hair, a soft robot system for hair care 💇🏻💆🏼 that uses mechanical compliance and visual force sensing for safe, comfortable interaction. Check it out: moehair.github.io 🧵1/7
Eliot Xing retweeted
Very Cool!
Thank you for bringing us this HUGE win!!!
RL is notoriously sample inefficient. How can we scale RL on tasks much slower to simulate than rigid body physics, such as soft bodies?
In our #ICLR2025 spotlight, we introduce both a new first-order RL algorithm, SAPO, and differentiable simulation platform, Rewarped. 1/n
Eliot Xing retweeted
RL for deformables!
RL is notoriously sample inefficient. How can we scale RL on tasks much slower to simulate than rigid body physics, such as soft bodies?
In our #ICLR2025 spotlight, we introduce both a new first-order RL algorithm, SAPO, and differentiable simulation platform, Rewarped. 1/n
Eliot Xing retweeted
Congratulations @etaoxing! This is a super cool use of Warp and differentiable simulation!
RL is notoriously sample inefficient. How can we scale RL on tasks much slower to simulate than rigid body physics, such as soft bodies?
In our #ICLR2025 spotlight, we introduce both a new first-order RL algorithm, SAPO, and differentiable simulation platform, Rewarped. 1/n