Shiva (@ShivaSujit) profile picture

Deep RL and Robots | Prev @ArayaGlobal, @MSFTResearch | MSc @Mila_Quebec in RL | BSc @ReachNITT

Joined May 2014
Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)Photo by Shiva (@ShivaSujit)
VLAs run 4–10 Euler steps per action. What if you just ran 1? Across 6 VLAs, 4 sim benches and 4.8k real-robot rollouts, 1 step largely matches multi-step, while being 1.6-4.8× faster, w/o ANY retraining or distillation. 📍 #CoRL2026 #111, Nov 9th 5pm (@lucanunz presenting)
3
5
1
15
1,038
Yeah these are the sorta lessons you get only from the trenches. Inspecting the data™ remains essential when testing robotic benchmark suites. @lucanunz & I have plenty of horror stories from trying to rigorously test models on RoboTwin
in our new blog, we take a look at RoboDojo sim, which are base on NVIDIA's Isaac Sim and it is a good robo benchmark used in Fable 5.2 model card. the video pretty much show what we found. the simulation is not even wrong. I'm now fully understand why robotics experts "hate" the past simulator. but from the first principle, I do believe LLM powered agent could do much much better simulation with the minimal but necessary real-world calibration is the right way we build embodied intelligence. real-world are slow, expensive, hard to scale, hard to get labelled, but sim not. then we partially solve the data problem for robotic and we could scaling up the compute!
1
1
11
833
Why it matters: integration steps increase latency on every replan. One step gives 1.6-4.8× faster action generation at nearly no cost to success. It could also make RL post-training cheaper: a single backward pass, no backprop through the ODE solver.
1
1
46
Robotics? Well yes, that also goes in the LLM shaped hole.
Another demo we did. This time I let Argon use the Spot. It went through the API, and even downloaded the SDK document to understand, and could quickly build a skill to pick up can and use it's foot to push the pedal to trash into a bin. Even it did the voice over to the video.
4
310
All will be revealed in due time 🤖
The @ArayaGlobal RL team will be presenting at #CoRL2026 🥳 In our work, we challenge a core assumption in modern robot learning, and show that maybe we're just overcomplicating things 🤷 Look forward to our study being released soon 🤘
2
6
694
The @ArayaGlobal RL team will be presenting at #CoRL2026 🥳 In our work, we challenge a core assumption in modern robot learning, and show that maybe we're just overcomplicating things 🤷 Look forward to our study being released soon 🤘
🎉 CoRL 2026 decisions are out! 687 papers accepted, a 32.8% acceptance rate across 2,094 active submissions. Thank you to everyone who submitted, and to the ACs, SACs & reviewers whose careful calibration shaped the decisions. 📣 Accepted authors: register ASAP! #CoRL2026
1
4
2
28
3,678
Great to see how many people are contributing to oopsie-data.com/! As the top contributor, at @ArayaGlobal we need to focus more on the quality of the submitted data and ensure that it is actually useful🤖
2
1
280
Look ma, we are at the top of the leaderboard. Didn't think it would be nearly half the dataset 🤯 @lucanunz 's weeks of data collection paying off lol
1
7
489
Data from oopsie-data.com They are open sourcing a dataset of policy rollouts, including successes and failures. A rich source of data for learning good value functions for RL (More from us on that soon 👀)
1
87
Shiva retweeted
we take this kind of plot for granted now, but RL compute scaling working on literally any domain (you can verify?) is still beautiful to me
I gave a talk at ICLR 2026 about how we are scaling RL on frontier LLMs with 1T+ parameters, on experimental data from our physical lab at Periodic! Here's a rough recording of the talk:
2
7
159
13,965
RL papers nowadays are about omitting just enough information to appear novel and ground breaking - @lucanunz Bro is already jaded after reading two (2) papers
3
162
Some progress on implementing RLT from @physical_int. Separated the online RL and encoder stages to test them individually. This is a simplified version of the ResNet baseline on a handover task in Aloha-Sim. Initial experiments look promising👀Similar curves to the paper.
1
2
191
Need to integrate interventions and async RL into the code, which should greatly improve sample efficiency. The policy often fails at transfering the cube, and some targeted interventions there should speedup learning.
1
76
Looks like a surprisingly simple method for fast finetuning of VLAs, with minutes (🤯) of data? Guess I know what the next paper Im going to pair program with Codex is 👀
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
2
281
Exciting news! 🚀 Araya and Fluent BCI are teaming up to revolutionize neural speech decoding 🤝 We're advancing "Speech Decoding BCI" to restore or replace lost communication abilities. Check out the thread below for the full story 👇
1
8
10
2,578
oh you're using VLAs? everyone's using GRPs now. just kidding we're all on LBMs. world models are the future so we developed our own WAM. we're using DVAs. we were using UWMs but our robot caught on fire so we switched to DreamUMVLAPs. we're shipping a robot that passes butter.
17
38
7
482
32,089