Kaiyuan Eric Chen (@ky_eric_chen) profile picture

Robotics PhD @UCBerkeley | Building the future of agentic robotics.

Berkeley, CA
Joined August 2022
Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)Photo by Kaiyuan Eric Chen (@ky_eric_chen)
We gave Claude and GPT a phone video of a robot wiping. In an hour, each built a full-dynamics digital twin. Introducing RoboSimStudio 🤖: an open and interactive toolkit for Agentic Robotics, enabling Fable, Astra and Argon AI agents to model sponges 🧽, cables 🧵 and cloth 👔.
40
53
21
455
183,712
Kaiyuan Eric Chen retweeted
From natural language to robot actions, fully on-device on NVIDIA Jetson AGX Orin. Nemotron + Graph-as-Policy + LeRobot SO-101. No cloud, fully local AI. Next: Sim-to-real deployment. #Dell #DellProPrecision #NVIDIA #Jetson #Robotics #PhysicalAI
2
3
16
817
Most exciting CoRL ever! We presented GaP as Spotlight and RoboSimStudio at Late Breaking results. RSS got 500+ github stars🌟 < 1 month and CoRL best paper finalist for PhysWM workshop. A thread on open source #AgenticRobotics 👇
4
4
2
85
5,047
We have also designed a variational automation benchmark for agentic robotics with @NVIDIARobotics. Our tool sorting, tool hanging, gear meshing, cable routing, cable insertion videos were proudly displayed at NVIDIA booth. Website: berkeleyautomation.github.io…
1
1
143
Also check out our recent blog on agentic robotics, which summarize our thoughts and future directions. berkeleyautomation.github.io…
1
91
Kaiyuan Eric Chen retweeted
Will LLMs eat robotics? Many say frontier labs will win by distilling models to run at real-time. The story is more complex, and one critical piece is simulation. Closing the sim-to-real gap creates a recursive loop for robotics models. I wrote for @substratemag: substratemag.com/turning-com…
14
24
7
185
14,538
Kaiyuan Eric Chen retweeted
Attending #CoRL26 in Austin, TX. We are presenting 3 papers. Main: @Bw_Li1024 is presenting our work on ReSYNC; @Shuangyu0_0Xie and @kaiyuan_c are presenting our work on Graph-as-Policy (GaP) Learn@Deploy workshop: @revvozz is presenting our work on REMM tomorrow at 10:30am.
1
2
10
637
Very exciting results that we will present today at CoRL. All motion is python code learned by agents and simulation transfers to real as well!
🚀 I’m presenting latest work at today’s CoRL Late-Breaking Results session, 12 PM at the JW Marriott! Learning from human demonstrations has long been a go-to approach for teaching robots complex motions. But what if robots could also learn complex behaviors through code generated by LLM agents? Checkout FlipAnything - a recent surprising finding on agentic robotics self improvement 👇
14
798
Thank you @NVIDIARobotics for featuring Graph-as-Policy (GaP) and our bar-wiping demo of RoboSimStudio! 🤖 Very excited to collaborate with #NVIDIA on VABAR, the Variational Automation Benchmark for Agentic Robotics! We'll present GaP as a spotlight talk at #CoRL2026 on Wednesday.
A finer touch for humanoid robots. 🤖 At #CoRL2026, NVIDIA is introducing GR00T 2, an open world action model for humanoids, alongside new open industrial benchmarks from @RLWRLD_ai and @UCBerkeley built on Isaac Lab-Arena. Explore how NVIDIA’s open world action models, scalable simulation tools and agentic workflows help researchers train, test and refine robot skills, from connecting cables to assembling small components. Discover more ➡️ nvda.ws/4ixvdZV
5
1
26
1,121
Wow thanks for highlighting RoboSimStudio, Vai! Really exciting to see how quickly frontier models are improving at Real2Sim. Still plenty of open challenges in physics accuracy and sim2real, but that's what makes this direction exciting!
1
1
16
977
Agentic Real2sim to the moon 🚀
Inspired by @johnzhangx's work, I wanted to see if we could bring agentic real2sim to space robotics! 🚀🛰️ Using footage of Canadarm2 (@csa_asc) operating aboard the ISS, I reconstructed the scene in simulation using RoboSimStudio with mjorbit support! robosimstudio.github.io/
11
625
Kaiyuan Eric Chen retweeted
Inspired by @johnzhangx's work, I wanted to see if we could bring agentic real2sim to space robotics! 🚀🛰️ Using footage of Canadarm2 (@csa_asc) operating aboard the ISS, I reconstructed the scene in simulation using RoboSimStudio with mjorbit support! robosimstudio.github.io/
introducing mjorbit: an open-source simulator for space robotics that runs thousands of worlds in parallel on a GPU. here, a single RL policy trained in mjorbit flies in from kilometers away in LEO, grabs a satellite with both arms, and points it at earth 🧵
1
5
2
22
18,613
Kaiyuan Eric Chen retweeted
In 2022, our first cuRobo prototype took 100 ms and consumed the whole GPU. Today, we’re open-sourcing cuRoboV2-Fast under Apache 2.0: collision-free, torque-limited motion planning in 4.4 ms. Collaborating with researchers across robotics and GPU architecture brought us to 50 ms, then 30 ms. B-splines, GPU time optimization, and faster kernels brought us to 4.4 ms while producing motions that execute faster. As planning got faster, perception, grasping, and control became the bottlenecks. We built a 7 ms perception pipeline (SceneFields): from stereo RGB to depth, voxels, and robot and object segmentation, and accelerated GraspGenX to 1.6 ms. We also integrated wrote a cpp controller for YAM and built a low-latency CUDA-GL visualizer for an interactive live view of the task. We deploy the stack as a two-process application: one dedicated to real-time control, the other handling perception, grasping, planning, and visualization. On a real YAM arm, pick-and-place cycles averaged 6 seconds, 34.6% shorter than cuRoboV2. A long journey, made possible by amazing collaborators. The rest of the stack is coming soon. research.nvidia.com/labs/lpr… Joint work with @Adithya_Murali_ , @BirchfieldStan Prompt your agent with : Use cuRobo cpp_runtime for collision-free motion planning: github.com/NVlabs/curobo/blo…
10
29
3
268
14,194
Kaiyuan Eric Chen retweeted
Each AI model has different strengths. Combining and harnessing them is currently more art than science but this will evolve, esp given new results on agentic weight tuning and RSI. Note how KEC combines Fable and Astra with RSS to rapidly tune complex simulation parameters for geometry, dynamics, deformations, materials, lighting, texture, and camera extrinsics to match a given input video. This fills a previous gap in Agentic Robotics ability to scale to new environments, objects, and tasks.
We recently found a new way to do agentic real2sim: using Claude Fable commanding GPT-6 Astra. Give it an online image, or even an Amazon link, and they collaborate to recreate the world in RoboSimStudio, ready for robots under 2 hours
8
9
1
116
595,427
Kaiyuan Eric Chen retweeted
So impressed by foundation LLMs skill at using RSS to tune complex robot manipulation sim parameters for geometry, dynamics, deformations, materials, lighting, texture, and camera extrinsics to match a given video.
Replying to @ky_eric_chen
Since we launched RoboSimStudio three weeks ago: 🏆 CoRL Late Breaking Results 🎤 Oral presentation at the PhysWM Workshop (speakers: @tom_erez @erwincoumans @ylecun @YunzhuLiYZ) 🌟 300 GitHub stars Thank you all 🙏
1
7
32
7,000
We recently found a new way to do agentic real2sim: using Claude Fable commanding GPT-6 Astra. Give it an online image, or even an Amazon link, and they collaborate to recreate the world in RoboSimStudio, ready for robots under 2 hours
13
14
3
98
647,213
Since we launched RoboSimStudio three weeks ago: 🏆 CoRL Late Breaking Results 🎤 Oral presentation at the PhysWM Workshop (speakers: @tom_erez @erwincoumans @ylecun @YunzhuLiYZ) 🌟 300 GitHub stars Thank you all 🙏
1
1
16
8,379
The paper, code, examples, and interactive 3D rollouts are all open 🌐 Site, 📄 Paper, 🎮 Demo: robosimstudio.github.io 💻 Code: github.com/robosimstudio/rob…
12
1,201