Minghuan Liu retweeted
GPT-6 Astra's robotic capabilities are not emergent. It was trained on robotics specific data. So were GPT-6.1 Sol and Gemini Argon
Here's how we know 🧵
Minghuan Liu retweeted
Wanted to get into manipulation. Decided to first build a full stack that scales.
Plan: copy @GeneralistAI's gripper. Fun weekend project.
It was not.
So @t_k_233 and I documented every design choice and lesson along the way.
Meet Specialist: fully open-source hardware, data pipeline, training, and deployment code.
specialist-ai.github.io/
We gave Claude Opus 5.5 a π₀.₅ checkpoint and one RTX 4090. It found a Training-Time RTC bug and a way to speed up robot actions via RoPE.
On a real YAM robot at the same 40 Hz, the edited policy cut bottle-placement time by 12% and reduced jumps between action chunks by 85%.
The weight edits approximate two code changes. One a bug in Training-Time RTC, the final denoising step can alter the reserved action prefix. The other widens action-token RoPE spacing.
The 30 Hz baseline took 22.42 s. The edited policy at 40 Hz took 15.83 s, a 29% reduction in task time.
See more details in
cuzyoung.github.io/robotics-…!
Opus-5.5 edits π₀.₅’s weights: ~30% less task time, 100% success in our bottle-placement trials.
We asked Claude Opus 5.5 to write code that edits π₀.₅’s weights and makes a YAM robot faster—no backpropagation, no production inference-code changes.
In ~111 minutes on one RTX 4090, it tested a RoPE-based speedup, fitted it into Q/K weights, and unexpectedly found and repaired an RTC prefix bug.
The ~30% reduction combines weight editing with a higher execution rate: original at 30 Hz → edited at 40 Hz. At matched rates, weight edits alone reduced task time by 4.3–12.0%.
We expected a few clever parameter changes. What unfolded was AutoResearch: hypotheses, experiments, failed attempts, and iteration.
Blog + robot demos:
cuzyoung.github.io/robotics-…
14
1,193
lmao
Just another Sunday afternoon in SF: I paid $60 for 2 humanoid robots to clean for an hour.
Here's how it went:
(1) What happened: Sergey and Chelsea (the names of the two robots) finished about 80% of one floor in an hour: tidying, wiping down surfaces, vacuuming, mopping, cleaning the toilet.
- They came with a staff engineer at the company, who dropped them off and picked them up after an hour.
- They were controlled by 1 human operator each, in the Mission where the company office is at.
- The engineer gave me a number to text them, I texted a request to wipe the mirror. They did it.
(2) Performance and dexterity: Chelsea dropped a toy, struggled with a pillow, and had trouble balancing on the playmat. But overall, for all the tasks they did, grippers were enough. 5 fingers were not needed.
- The engineer told me the hardest task for them was making bed.
- They were quite loud, stomping around the house. Took a bit of time to get used to. They both wore gripper socks which protected the floors and helped them not slip.
- Interaction: I didn't know they can respond via voice. In the end they were conversing with us in voice which was nice.
(3) Unexpected observation: Their pants and shirts have holes in them. It's because they were dragged on and off the vans that transported them.
(4) Other user feedback: I shared my experience with some friends afterwards.
A user told me that unloading dishwasher was an "advanced task". They didn't attempt at mine since it was running at the time they were working. Another user said they scratched their hardwood floor. That's probably why they now wear gripper socks.
(5) Did they do as well as human cleaners? In terms of the tasks they did, I think so. In terms of speed, they were ~80% speed of humans when it comes to the more delicate tasks, but I would say they are quite smooth. They didn't finish the whole place, so it wasn't a full service.
(6) How does the math work out?
Each booking was only for 1 hour, since their batteries need to be swapped after 1 hour. So they didn't finish the place.
For a user: In SF, it's ~$200-220 to clean a full home around my neighborhood, with two human cleaners. Usually it takes around 2.5 hours. So around $40/manhour. This is $30/robot hour. Slightly cheaper, but again it's not fully finished with the 1 hour they were available.
For the company: It took 2 robots, 2 human operators, 1 staff member to dropoff/pickup, and 1 driver of the van to transport the robot. Definitely labor intensive. But they also retain the data they collect for training purposes.
(7) My biggest takeaway as a user: interaction / hospitality mattered. One of the humanoids was more friendly (Sergey), he waved at us when he came in. It'd have been great if they both introduced themselves in the beginning.
Beyond that, being able to do 2 hour slots (instead of one) would be great to finish the cleaning.
Overall, I appreciated their work and the fact that they got the robots out in people's homes. Deployment is not easy and so much learning from it each time
262
While GPT-6-Astra is making everyone anxious, we’ve been working on the System 1 side of robotics: timely responses, continuous motion, and efficient execution.
I think a robot’s intelligence also shows in its ability to keep pace with a changing world. Perception, inference, and control operate at different rates and with different latencies. New observations keep arriving, inference takes time, and actions need to keep moving forward. Coordinating these asynchronous streams—so a robot can act smoothly despite latency and adjust as new information arrives—is a problem worth studying carefully.
Real-time performance takes work across the whole system. Model capabilities, inference strategies, and execution mechanisms have to come together in a continuous feedback loop. I’d love to see more real-robot experiments and discussion around these questions.
Proud of our team for putting over two months of work into ManiMux, now open source. I hope these tools make it easier for more people to deploy their robots and test their ideas. Give it a try—contributions welcome!
Robotic manipulation is not only about success rate. It is also about getting things done efficiently.
The real world runs in real time, asynchronously. It does not pause for model inference. Turning model capabilities into smooth, continuous, efficient robot behavior requires the right inference and execution strategies—from handling latency to transitioning between action chunks. From early work on asynchronous inference in Diffusion Policy to π’s @physical_int later work on RTC, the robotics community has paid increasing attention to these challenges.
As robot foundation models become available through cloud APIs, coordinating inference and execution becomes even more critical. The release of the Dream-1 API @dreamlabs_ai gave us a timely opportunity to explore this. With several official demos shown at 8× speed or faster, we wanted to see: How does Dream-1 perform in real time? And how much could running inference and execution asynchronously improve motion continuity and task efficiency?
Our team quickly integrated Dream-1 into ManiMux, our in-house framework, and tested it on YAM.
The results surprised us: even with seconds of cloud API latency, better asynchronous inference and execution scheduling improved fine-manipulation performance and efficiency. In our block-stacking tests, asynchronous inference doubled Dream-1’s task completion efficiency!
Increasingly capable System 2 models are exciting. The System 1 side of robotics—timely responses, continuous motion, and efficient execution—deserves attention too. We want to make these capabilities easier for the community to use, compare, and reproduce.
While GPT-6-Astra @OpenAI is making everyone anxious, we want to share a few insights beyond System 2—and make it easier for everyone to build a fast, responsive System 1 for real robots.
That is why we are building ManiMux, an open-source inference harness and experiment management platform for real robots, designed to make deployment, inference, and evaluation easier and more reproducible.
ManiMux currently integrates policy frameworks including XPolicyLab and StarVLA, inference algorithms including RTC, PAINT, and AutoHorizon, and robot platforms including Tianji, YAM @i2rt_robotics , and AgileX @AgilexRobotics . Support varies across model, algorithm, and robot combinations, and we are continuing to expand real-robot validation and improve usability.
To make integration easier, the repo also includes comprehensive agent documentation and Skills. Use your coding agent to quickly integrate your own robots and policies, adapt deployment workflows, and build new features—so you can spend more time experimenting on real robots.
We have been developing ManiMux for over two months. We are also collaborating with teams around the world on a systematic comparison of inference algorithms, with more observations and practical recommendations from real-robot experiments to come. Stay tuned.
Try ManiMux, open an issue, contribute code, or share your real-robot experience. Let’s make robot experiments more reproducible—and robot execution smoother and faster.
Let’s make something meaningful, together.
Github: github.com/SII-LiuLab/manimu…
The original Qwen 3.8 27B is not quite good for captioning ego and robot data. This fine-tuning is super useful. Can’t wait to try!
We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals.
Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too.
Flora gives you:
- High-quality, timestamped captions to curate and filter human egocentric and robot data;
- Dense progress rewards for quality control and RL post-training;
- Rubrics scoring for real-world benchmarking
Try it today, links in thread:
7
1,223
I experiment with a lot of different types of robots and the conclusion is similar for people's info. And the more complex the robot is, the more advantage RL has.
We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals.
Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too.
Flora gives you:
- High-quality, timestamped captions to curate and filter human egocentric and robot data;
- Dense progress rewards for quality control and RL post-training;
- Rubrics scoring for real-world benchmarking
Try it today, links in thread:
Minghuan Liu retweeted
following the discussion weeks ago, I asked codex to vibe a comparison for MPC vs RL for just table-top EE tracking, 1.6Mtok gone. It uses Pinocchio for MPC, and codes a cupy-based simulator, tuned RL to achieve better tracking accuracy (actually both errors are too small)
Minghuan Liu retweeted
Can a robot assemble tight-clearance parts using only its proprioception? No camera, no tactile sensor, no F/T sensor, no joint torque sensing, no priors over the object geometries or poses: just joint position readings.
It actually can! Blind policies trained in simulation insert pegs, mesh gears, and thread nuts, and the peg policy works zero-shot in the real world. Here's what we found 🧵
Joint work with @BronarsToni, my wonderful co-first author, who's on the job market right now! Advised by @kushalk_ and @pulkitology.
I finally baked a long-form audit of “robot scaling laws.”
AI agents did a lot of the digging; I did a lot of checking, rewriting, and arguing with the drafts.
I reviewed 19 public results related to robot scaling. The short version:
scaling curve ✅
scaling law ❌
Here is the evidence funnel:
19 public results reviewed
→ 16 report multi-scale curves
→ 13 keep the main variable and evaluation reasonably controlled
→ 1 tests a held-out loss forecast
→ 0 completes the full loop from fitting loss, to forecasting the next scale, to validating that forecast on real robots from the same model family.
This distinction matters.
In LLM scaling experiments, data volume D is usually measured in tokens. Token count does not tell us how much new information the data contains, but it remains a relatively stable training unit when the tokenizer, data pipeline, and distribution are held fixed.
Robotics does not yet have an equivalent common unit. Different studies use recorded hours, trajectories, frames, action steps, or action tokens. Even the same “one hour” can produce very different training examples under different sampling rates, action representations, chunking schemes, filtering rules, tasks, environments, and embodiments.
The reported objectives and y-axes vary even further: held-out denoising or vector-field MSE, decoded action error, thresholded accuracy, simulated progress, task completion, cycle time, and real-robot success all answer different questions.
A few examples:
— Figure Helix 2.5 provides the only held-out loss forecast I found: the first three Index-data scales are used to predict offline action loss at 8×. But its 9%→56% real-robot result comes from a separate with-vs-without-pretraining experiment, not the endpoints of that loss curve.
— Dyna-2 scales egocentric human video from 1k to 1M hours. Offline [email protected] moves from roughly 40% to 47%, while post-trained robot performance rises from 20% to 53% on its normalized metric. Scaling helps, but the public results cannot tell us whether the final 900k hours were worth the cost.
— Xiaomi-Robotics-1 reports that, with a fixed 5B model, increasing UMI data from roughly 2.5k to 20k hours raises mean success from 53% to 75%. At fixed data, moving from 5B to 10B adds only four points, from 75% to 79%. Useful evidence, but the full model × data grid is still missing.
— Sunday ACT-2 shows that a quality-selected 12.5% subset can outperform a uniformly sampled 50% subset. Nominal scale is not enough: data distribution, repetition, and coverage can matter more than another hour of collection.
My current view:
A general law spanning tasks, environments, and embodiments will probably arrive later, because robotics still lacks a shared training and validation distribution.
But deployment may not need to wait for a predictive scaling law. If a system is reliable enough on a concrete task at an acceptable cost, products may arrive before a general law does.
Perhaps robotics is not waiting for a GPT moment, but an iPhone moment: hardware, software, data, and intelligence finally coming together in a product that works reliably in the real world.
Full visual audit, original figures, evidence boundaries, and sources:
minghuanliu.com/robot_scalin…
Can a humanoid learn robust whole-body loco-manipulation directly from a human? No robot, no teleop, no retargeting.
Meet Workhorse 🐴: our G1 topples and climbs a suitcase, catches a thrown box, and sorts boxes with hands and a kick. Fully autonomous.
More videos at hybridrobotics.github.io/wor…
Localizing UMI using SLAM is so annoying 😫
I’ve been collecting robot data with UMI.
The idea is incredibly powerful:
- collect manipulation demonstrations without needing a robot.
BUT actually getting good post-training data isn't so easy!
Here are some lessons from using UMI 🧵
The speed is seriously impressive — I can only imagine how much hard work went on behind the scenes. Huge congrats to the team on the release!
Introducing OM-1, our first robot foundation model: zero-shot generalizing across table-top arms, industrial arms, and humanoids.
- Learned directly from human manipulation data
- No teleop or robot data
- Near human-level dexterity and efficiency
- Multi-robot collaboration
Looks amazing
Meta just open-sourced its dexterity stack! 🪬
Most physics simulators were built for things that move through space.
Walking robots, drones, cars. Contact is the part they approximate worst, and obviously dexterous manipulation is nothing but contact.
Project SuperDex, from Meta Reality Labs Research, is built the other way around, a contact-first physics engine with the whole platform stacked on top of it.
The cool part is that it's on GitHub.
The engine runs one solver across rigid bodies, soft bodies, rods and tendons, shells and cloth, in the same model.
→ Non-convex collision with accurate contact force distributions, so a multi-finger grasp gets simulated rather than approximated
→ Tactile sensors and soft contact as first-class primitives
→ Numerical stability without the tight time-step limits explicit solvers force on you
→ Constraint-aware inverse kinematics running on the same optimization core as the forward dynamics
Then the data layer. Put on a Quest 3, teleoperate the simulated hand with haptic feedback, and generate demonstration datasets without touching real hardware.
They show a shape-sorting policy trained entirely in simulation and deployed zero-shot on a real robotic hand.
Robot hands are getting good. Data for them isn't that fast.
Meta is betting the cheapest way to collect contact-rich demonstrations is a headset people already own, pointed at a simulator instead of a game.
🔗 Here's the project page: projectsuperdex.com/
~~
♻️ Join the weekly robotics newsletter, and never miss any news → ziegler.substack.com
6
1,656
Minghuan Liu retweeted
🚢 Marin 535B-A23B started training this week! As usual, the whole process is open.
Voyage plan: pretraining (80%) + midtraining (20%) on 18.75T tokens on 11 x GB200 NVL72 for ~3 months (2.7e24 FLOPs). Post-training will follow.
Before kicking off the run, we trained a 4-rung scaling ladder from 1.6B-A61M (48B tokens) to 27.7B-A1.2B (926B tokens) to debug issues, and to make a forecast of our hero run. This is by far our biggest run, so definitely expecting the unexpected.
Minghuan Liu retweeted
Good question thanks Ani. Here's a video that gives a sense of the answer here about the effect of prompting:
- in the first part, the model is just babbling with no prompt in context
- at the end, we add a prompt for taking money out of a pouch (generalizing to the wallet)
Replying to @peteflorence @andyzengineer
Very cool work, Pete! I'm curious if you've tried an extreme version of this: no in-context example at all; just place objects in front of the robot and see if it can infer the task.
I suspect this will have non-trivial success rates, which would also allow you to figure out how much of the benefit is actually coming from the prompt (rather than pure zero-shot task inference + capability).
I remember Russ showed something like this in his Stanford talk on LBMs last year:
youtu.be/TN1M6vg4CsQ?si=-8J-…



















