
@stepjamUKi
iAccount based inUnited Kingdom
About this account
- Account based in
- United Kingdom
- Connected via
- United Kingdom Android App
Account-level information from X, not a live location or the device used for a specific post.
CEO @Neuracore_AI | Assistant Professor @imperialcollege | ex-Director of Dyson Robot Learning Lab | Postdoc @UCBerkeley w/ @pabbeel | PhD ICL w/ @ajdDavison
London, England
Joined January 2010
- Tweets568
- Following242
- Followers7.7K
- Likes911
Pinned Tweet
๐๐ณ๐๐ฒ๐ฟ ๐ญ๐ฌ+ ๐๐ฒ๐ฎ๐ฟ๐ ๐ถ๐ป ๐ฟ๐ผ๐ฏ๐ผ๐ ๐น๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด, from my PhD at Imperial to Berkeley to building the Dyson Robot Learning Lab, one frustration kept hitting me:
๐ช๐ต๐ ๐ฑ๐ผ ๐ ๐ต๐ฎ๐๐ฒ ๐๐ผ ๐ฟ๐ฒ๐ฏ๐๐ถ๐น๐ฑ ๐๐ต๐ฒ ๐๐ฎ๐บ๐ฒ ๐ถ๐ป๐ณ๐ฟ๐ฎ๐๐๐ฟ๐๐ฐ๐๐๐ฟ๐ฒ ๐ผ๐๐ฒ๐ฟ ๐ฎ๐ป๐ฑ ๐ผ๐๐ฒ๐ฟ ๐ฎ๐ด๐ฎ๐ถ๐ป?
๐ง๐ต๐ฒ ๐ฝ๐ฎ๐๐๐ฒ๐ฟ๐ป ๐ ๐ธ๐ฒ๐ฝ๐ ๐๐ฒ๐ฒ๐ถ๐ป๐ด:
โข New robotics team starts
โข Spends 6 months building data collection pipeline
โข Spends another 3 months debugging synchronization issues
โข Finally starts collecting task-specific data
โข Realizes their infrastructure choices limit their flexibility
โข Starts over
๐ง๐ต๐ถ๐ ๐ถ๐ ๐๐ต๐ฒ ๐๐ต๐ผ๐น๐ฒ ๐ฝ๐ผ๐ถ๐ป๐ ๐ผ๐ณ ๐ฟ๐ผ๐ฏ๐ผ๐ ๐น๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด: Robot learning is fundamentally data-driven. Whether you're picking strawberries or assembling electronics, the core infrastructure needs are identical. That's actually why I was so interested in pursuing data-driven robotics over a decade ago.
๐ฌ๐ผ๐ ๐ฎ๐น๐๐ฎ๐๐ ๐ป๐ฒ๐ฒ๐ฑ:
โข Multi-sensor data synchronization across different frequencies
โข Flexible storage that works with future algorithms
โข Visualization tools to understand your data
โข The ability to experiment with different temporal resolutions
โข Robust logging that captures everything you might need later
The trend towards AI in robotics is growing, with robots needing to process and analyze large amounts of sensor data to manage variability and unpredictability in real environments.
๐๐๐ ๐ฒ๐๐ฒ๐ฟ๐ ๐๐ฒ๐ฎ๐บ ๐ฏ๐๐ถ๐น๐ฑ๐ ๐๐ต๐ถ๐ ๐ณ๐ฟ๐ผ๐บ ๐๐ฐ๐ฟ๐ฎ๐๐ฐ๐ต. Imagine if every web developer had to build their own database, web server, and deployment pipeline before writing their first line of application code.
๐ง๐ต๐ถ๐ ๐ถ๐ ๐๐ต๐ ๐ ๐ณ๐ผ๐๐ป๐ฑ๐ฒ๐ฑ ๐ก๐ฒ๐๐ฟ๐ฎ๐ฐ๐ผ๐ฟ๐ฒ.
Instead of every robotics team spending months on infrastructure, we provide the common tools that let you go from "I have a robot" to "I'm shipping intelligent robot behaviors" in days, not months.
๐ง๐ต๐ฒ ๐ฟ๐ฒ๐ฎ๐น ๐ถ๐ป๐ป๐ผ๐๐ฎ๐๐ถ๐ผ๐ป ๐ถ๐ป ๐ฟ๐ผ๐ฏ๐ผ๐๐ถ๐ฐ๐ ๐๐ผ๐ป'๐ ๐ฐ๐ผ๐บ๐ฒ ๐ณ๐ฟ๐ผ๐บ ๐ฒ๐๐ฒ๐ฟ๐๐ผ๐ป๐ฒ ๐ฟ๐ฒ๐ฏ๐๐ถ๐น๐ฑ๐ถ๐ป๐ด ๐๐ต๐ฒ ๐๐ฎ๐บ๐ฒ ๐ฝ๐น๐๐บ๐ฏ๐ถ๐ป๐ด. ๐๐'๐น๐น ๐ฐ๐ผ๐บ๐ฒ ๐ณ๐ฟ๐ผ๐บ ๐๐ฒ๐ฎ๐บ๐ ๐๐ต๐ผ ๐ฐ๐ฎ๐ป ๐ณ๐ผ๐ฐ๐๐ ๐ฒ๐ป๐๐ถ๐ฟ๐ฒ๐น๐ ๐ผ๐ป ๐๐ต๐ฎ๐ ๐บ๐ฎ๐ธ๐ฒ๐ ๐๐ต๐ฒ๐ถ๐ฟ ๐ฎ๐ฝ๐ฝ๐น๐ถ๐ฐ๐ฎ๐๐ถ๐ผ๐ป ๐๐ป๐ถ๐พ๐๐ฒ.
Robot learning shouldn't be bottlenecked by infrastructure. It should be bottlenecked by creativity.
What's the longest you've spent building infrastructure before getting to the actual robotics problem you wanted to solve?
Two big things here:
1) BiGym 2.0, a new whole-body humanoid benchmark. 2) The first thorough comparison of learned policies vs LLM-written robot policies.
I was surprised how far coding agents got from just 1 demo, where learned policies would be useless. But ultimately, learned policies (from scratch or VLAs) are the more general solution, handling a broader range of tasks, albeit with more demos.
Coding agents can now write a humanoid's controller from one demo. So are learned policies dead?
We built BiGym 2.0, a household loco-manipulation benchmark, to find out.
Short answer: not yet. GPT-6 Astra hits 65% with 1 demo, but ฯ0.5 trained on 60 demos still leads at 75%, especially on precise object rearrangement. ๐งต
Wrist cameras help robots a lot. Taking them off and letting the robot look at the right thing helps more.
EyeRobot 2.0, from Kush Hari, @justkerrding and the team at @UCBerkeley and Amazon FAR (advised by @Ken_Goldberg and @akanazawa), has no wrist cameras at all. One stereo head moves its gaze to whatever matters at each step. Sharp in the middle, blurry at the edges, like our own eyes.
Seven real tasks, including capping a marker, zipping a bag and pulling a pan out of a toaster oven. Same demonstrations and same policy architecture throughout, 25 trials per task.
Fixed stereo camera: 27%. Wrist cameras added: 52%. Active gaze: 67%.
When a grasped tool blocks the wrist cameras, that policy drops to roughly the level of the fixed camera. Gaze keeps working.
The gaze is trained with RL, with no gaze demonstrations.
Where it looks is where it reaches. When the gaze lands on the wrong object, the robot picks up the wrong object. That makes failures easy to read, and it suggests where to improve next.
Paper: arxiv.org/html/2610.03710v1
Source: eyerobot2.github.io/
#PhysicalAI #RobotLearning #VLA
Uni-VLaT adds touch to pre-trained VLA policies for humanoid loco-manipulation, and the Back-Tap Walking result is the one that actually got my attention. Tap the robot's back, it walks. Stop tapping, it stops. Zero visual signal involved, that's the whole task. 0% success without touch, 85% with it, which isn't really a "touch helps" result, it's a "this task doesn't exist without touch" result.
The rest of the numbers are more the usual incremental story, 68% to 75% average across five tasks from adding a prediction objective on top of raw tactile input, which is a real gain but not a dramatic one on its own.
Also generalizes across GR00T and ฯ0.5, so at least it's not a party trick tied to one backbone.
Paper: ggkiller-air.github.io/Uni-Vโฆ
Source: arxiv.org/html/2609.35450v1
Credit: @Tsinghua_Uni, @Beihang1952, @HKUniversity, @CUC1954_
Some robot tasks are basically impossible for a human to teleoperate cleanly. Not hard, impossible. Carrying a plate across a table with two independently controlled arms, for instance. Zero percent success rate, no matter the operator.
GLIDE takes an interesting approach to this. An LLM reads the task, predicts where a human will fail, and writes a filter that sits between commands and robot to prevent it. Watches the data, catches what's still failing, rewrites itself.
Same task, 0 to 70 percent. Two others go from 10 and 0 up to 90 and 90.
Keep the guardrail running at deployment and a policy that barely worked alone jumps from 0 to 60-70 percent too.
A good reminder that a lot of "the robot can't learn this" is actually "no human could demonstrate this in the first place."
Source: guardrail-policy.github.io/
Most robots need hundreds of demonstrations to learn a task, then need hundreds more the moment anything about that task changes. This one needed one, and then just kept adapting.
The system is VLBiMan++, out of @ShenzhenUni and @DexForceAI. Show it a single human demo of pouring water, and it breaks the task down into reusable pieces, then uses vision-language grounding to figure out how those pieces need to shift when the situation changes. No retraining, no extra demonstrations, no fine-tuning.
The range of what "changes" means here is the part worth watching. It's not just moving a cup to a new spot on the table. The same one-shot prior carries across pouring, reorienting and unscrewing a cap, spoon and funnel tool use, zipping a pen bag shut, coiling a loose cable, folding a towel. Rigid objects, articulated ones, deformable ones, all handled by the same underlying approach, each needing its own way of tracking what "the object" even means as it changes shape.
Then there's the embodiment jump. Skills learned on one dual-arm platform transfer directly to a completely different humanoid-style dual-arm robot, plugging in a pen, handing off an object, pouring, unscrewing, all without a single new demonstration on the new body. And to check it isn't just lucky on the first try, they ran the same task, uprighting a bottle knocked onto its side, twenty times in a row on the new platform and it held up.
Every added axis here (new tasks, new objects, new scenes, new robots) is a dimension where most systems need fresh data. This one just needs to reason more carefully about what actually changed.
Video: hnuzhy.github.io/projects/VLโฆ
Paper: arxiv.org/abs/2609.14310
Rhoda AI checked whether scaling web video pretraining actually helps real robots, using a real customer task. Unpacking 10kg boxes of bearings and sorting the packaging.
Turns out yes. Bigger models do better, more pretraining compute does better, and the compute gap is widest when you don't have many robot demos.
Refreshing to see a company display this amount of rigour. Rare these days.
Link to their blog post: rhoda.ai/research/scaling-weโฆ
A robot's camera predicts the future to decide what to do next. The problem is that prediction takes time, and time is the one thing a robot moving in the real world doesn't have to spare.
That's the tension a team from @BAAIBeijing and collaborators went after with World Action Models, systems that use video generation to imagine what happens next before deciding how to act. The video model builds that prediction through several rounds of denoising, and each round adds latency to the control loop. The obvious fix is to cut it down to one round. The team found out why that quietly breaks things.
Watching the denoising process closely, they noticed the background of a scene sharpens almost immediately. The gripper, the object it's about to touch, and how the two interact stay blurry until several steps later. Cut the process short and you keep a crisp picture of the room and lose exactly the part the robot needed to act on.
Their fix, called DIDO, trains a single denoising step to hold onto that missing detail. A slower four step model acts as a teacher, and the one step model is explicitly guided to track the object, the gripper, and their interaction directly, with extra supervision pointed at exactly those regions rather than the whole frame. A pruning step then spends compute where the scene is actually changing and compresses the parts that aren't.
The result is a model that predicts an action in 384 milliseconds instead of 562, while doing just as well or better on manipulation benchmarks; 99.0 percent on LIBERO, and the strongest score among methods with no embodied pretraining on RoboTwin. It also held up on a real Galbot G1 arm across four tasks, including with clutter and repositioned objects added in.
It's a good reminder that not every shortcut is free. Cutting a process down to save time can end up cutting away the part that made it work in the first place, and the only way to know is to look closely enough to see what's actually happening at each step.
Paper: arxiv.org/abs/2609.15570
Project page: loveju1y.github.io/DIDO/
Try holding your arm perfectly still for two minutes while someone else moves it into a sleeve. Now imagine doing that if holding still wasn't something your body could easily do in the first place.
That's the gap in a lot of robot-assisted dressing research, which usually assumes a static arm. A paper currently under review takes this on directly. It fuses visual and force feedback into a single policy, trained first in simulation and then fine-tuned with real-world data, so the robot adapts to natural, uncooperative arm movement instead of requiring the person to stay put. It also handles long-sleeved garments, which add friction and occlusion that shorter sleeves don't.
The results are the part that stands out. Tested across 12 participants and 264 real dressing trials, the system successfully dressed an average of 85 percent of each person's arm length, despite the arm moving throughout.
This is the same principle behind a lot of the manipulation research worth watching right now. Fusing force and vision into one policy, rather than treating them as separate systems bolted together, tends to be what makes robots reliable outside a controlled lab setting.
It's still under review, so treat the numbers as early rather than final, but the direction is one worth following.
Paper: arxiv.org/html/2609.04759v1
Source: anonymous.4open.science/w/drโฆ
RT @Neuracore_AI: There's about $50-60 billion worth of electric motors in use in industrial robotics today. To make the more extreme prediโฆ
Most of the robots sitting on factory floors right now are more capable than the tasks they're being used for.
I've spent over a decade in robot learning, PhD at Imperial, postdoc at Berkeley, building the Dyson Robot Learning Lab, and the pattern hasn't changed. Every major step, from early imitation learning to today's VLAs and diffusion policies, was deployable years before most teams picked it up. The capability was never the bottleneck.
The hardware in front of most companies right now can already do far more than it's doing. What's missing isn't a robot arm or a breakthrough model, it's the data and infrastructure to teach the hardware they already own new skills.
That gap is what's costing teams the opportunity in front of them. Not a lack of technology, a lack of the layer that turns existing hardware into new capability.
That's the layer @Neuracore_AI builds.
Want to see what your hardware could actually be doing? Head to the comments to book in a demo with our team.
Teaching a quadruped robot to push an object it can't grasp is a hard RL exploration problem. The task reward stays at zero until contact happens, so a standard single-critic PPO ends up optimizing smoothness and energy penalties instead, and never finds contact at all.
Members from @Unipisa, ETH Zรผrich & @nvidia address this with a separate exploration critic trained on a dense contact-seeking reward, guiding the end-effector toward candidate contact points, then decaying its weight over training so the policy shifts from contact-seeking to task-optimal once the physics has actually been found. Candidate points come from a general-purpose grasping algorithm, so the approach generalizes across object geometries without hand-tuning per task.
Validated on a real quadrupedal mobile manipulator transporting chairs, transferring zero-shot to unseen IKEA furniture, recovering from failed contact attempts, and remaining stable under loads beyond the robot's rated capacity.
A clean approach to a common problem: rather than hand-shaping one messy scalar reward, split exploration from task performance and let one decay into the other.
Paper: tolomeis.github.io/contact-gโฆ
Project page: tolomeis.github.io/contact-gโฆ
Skill learning is not new.
At Automate Further, I briefly walked through a decade of work in this space, from early imitation learning with hundreds of thousands of grasps, through sim-to-real transfer, to reinforcement learning agents that can start improving from just a handful of demonstrations.
This has been building for years, and itโs deployable today to solve real-world problems.
Head to the comments for a link to the full talk.
#PhysicalAI #RobotLearning
Stephen James retweeted
Building aircraft at scale means solving the same automation problems everyone else does, just with a lot less room for error.
At Automate Further, Marco Chacin, Head of Actuators and End Effectors at @Airbus, spoke about the realities of deploying robotics in aerospace manufacturing: competing for robotics talent against tech companies, and working within strict ISO and regulatory standards to do it safely.
Watch the full talk here:
youtu.be/yYQnzhqjoZE?si=-Q2Nโฆ
#AutomateFurther #Robotics #Aerospace
Stephen James retweeted
Every manufacturer asks the same question before automating: will this pay for itself?
At Automate Further, Ross Lacy, Sales Director at @rarukautomation, talked through why answering that question clearly, and keeping deployment simple, matters more to UK manufacturers than any new piece of robotics tech.
Watch the full talk here:
youtu.be/Uwnp9OYYqkY?si=yaQ3โฆ
#AutomateFurther #Robotics #Automation
Stephen James retweeted
"The bottleneck isn't capability. The bottleneck is adoption."
That's the case Laurie Barnes MIET, CTO of @theautomateuk, made at Automate Further. Not a technology problem. A skills, funding and confidence problem across UK manufacturing.
Watch the full talk here: youtu.be/v90uIHpJ_v4?si=0P6Lโฆ
#AutomateFurther #Robotics #UKManufacturing #Automateuk
Stephen James retweeted
Missed out on Automate Further?
Our founder and CEO, @stepjamUK, took the stage to walk through the real history of robot skill learning, from imitation learning at Google to todayโs VLAs and diffusion policies.
The full talk is now live on our YouTube, with the rest of the speaker presentations going live throughout the week.
Watch it here: youtube.com/watch?v=Hq8E2aq4โฆ
#RobotLearning #PhysicalAI #Robotics #AutomateFurther
FANUC, @ABBRobotics and @Universal_Robot arms trained on @Neuracore_AI, running autonomously all day at our first event.
At Neuracore, we believe in showing, not telling.
So at Automate Further, we put the platform to work: cable picking, cable wiring and timing belt fitting, all running autonomously on real hardware.
Here, a FANUC arm is fitting a timing belt onto a pulley. No script. No waypoints. The entire process, from collecting demonstrations to training and deploying the task, was done on Neuracore.
We just wrapped our first event at @Neuracore_AI!
Thank you to everyone who turned up and kept the conversation going long after the last talk.
Make sure to follow the Neuracore page to keep in the loop for more events like this, and check out the discussion over on our YouTube, coming later this week!
That's a wrap on Automate Further at Neuracore HQ!
Thank you to everyone who came down, filled the room, and stuck around afterwards to keep the conversation going.
Huge thanks to our speakers from Fanuc, @rarukautomation, @Airbus, @HALrobotics, @ARIA_research, @theautomateuk, @the_MTC_org & Fanuc for an open discussion on how we can close the gap in UK robotics.
ICYMI or couldn't make it, the talks will be live on our YouTube channel this week.
Follow us to stay in the loop and be first to hear about future events.







