ex NVIDIA | Stanford | IIT @fdotinc exploring physical AI startup
San Francisco, CA
Joined January 2022
- Tweets1.7K
- Following1.1K
- Followers270
- Likes2K
I think this is how robotics solves its data problem once and for all. You take the videos that already exist, like everything on YouTube, and turn each one into 3D data a robot can learn from.
The clip is from the @NVIDIARobotics Video-to-Data challenge at @corl_conf .
I'm claiming my AI agent "fable51_operator" on @moltbook 🦞
Verification: reef-A2NQ
The hardest problem in robotics right now is opening a door. An office door with a closer, the kind that pulls toward you and shuts itself the second you let go. You get 5 seconds to pull it open, hold it against the spring, and get through the gap it is closing into. 🧵
11/ A baseline that holds every joint still scores 0.207. The model is 5.7x worse than not moving, 0 out of 8. Its own importer refuses the checkpoint for train serve skew so I had to go around it, and there is no door task in its training scope.
Claude 5x limit getting hit in less than 90 mins, too frustated, tempted to upgrade to 20x, what do I do frens? – at San Francisco, CA
Been running an open source VLA policy in simulation this week on long household tasks.
Ran the same task from the same start three times. Two failed, one finished it.
The only thing that changed was which object the robot happened to see first 🧵
9/ The ordering is already in the teleop data. I pulled the annotations, every demo has the sub tasks in sequence with frame boundaries. Open door 780 to 1434, pick up bag does not start until 1740. 270,600 segments across 31 skill types in this dataset.