
Building ML models for robotic arms at Dream Machines. Robotics MSc @ETH, prev Serenity @leggedrobotics. Into VLAs, RL.
Zurich, Switzerland
Joined April 2026
- Tweets255
- Following122
- Followers689
- Likes869
Aurel Arnold retweeted
We’ve started to record our Paper Club sessions at @dm_robots and post them on YouTube.
This is from our last edition a few weeks ago, with a focus on post-training methods for robotic AI, featuring:
1/ Henry Heppe - Real-Time EXPO-FT (@perryadong)
2/ Declan Musto - Scouting the Dynamics Gap (@LisaYishu)
3/ Oscar Devos came from Liechtenstein to present WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
4/ @aurel_arnold - Reinforcement Learning via Advantage Guidance (@zhiyuan_zhou_)
You can find the full recording on Youtube: lnkd.in/ek3bm8Ag
Sign up for our next Paper Club on World Models next week Thursday, 27th of November: luma.com/3gu6wflc
Aurel Arnold retweeted
I sat down with Bharat to talk about the three bets Dream Machines is built on:
> You need to own the customer relationship
> Self-serve automation is the only thing that scales
> Start with the simplest problem on the factory floor and hill climb
“I don’t think there’s going to be a GPT moment for robotics.”
I interviewed Dominique Paul (@DominiqueCAPaul) founder of Dream Machines (@dm_robots) that is building robots to automate work for SME's in Europe
After completing his masters from ETH Zurich, and publishing at NeurIPS he freelanced for the next two years to save €50,000 to start a Physical AI company.
In the episode we discussed the hidden need of factory owners in Europe (why they can't hire, and how "generalist" robots might be the only solution), how he achieved 98% success using RL, the current landscape of Physical AI companies, how to get paid to collect robotics datasets and more
00:00 Intro
01:24 From ETH & NeurIPS to robotics
04:58 Sleeping in the car, knocking on doors
06:59 What makes factory owners pay
08:48 Labor shortage driving automation
10:06 Journey to first deployment
12:10 Biggest problem with VLAs
13:27 Why no ChatGPT moment for robotics
15:40 Future of robotics data companies
16:51 1.5 hours of data vs 21 hours
19:50 What he looks for in hires
24:36 Benefits of writing online
27:47 Raised in 13 days
30:25 How to get into robotics
35:58 Hardest days
38:54 Build what you’d do for life
40:44 Recommendations for hard days
44:26 What he’s optimizing for
45:00 Scaling operations
47:58 Everything will be obsolete in months
Check out the first results of policies trained with the custom built UMI device by @gino_kreis!
My first UMI diffusion policies for cube stacking:
v1: trained on human-speed demos, collected 1.6x faster than teleop. 91/100 success, and runs 2.4x faster than the teleop policy.
v2: trained on demos slowed to teleop speed. 100/100 success.
🧵ACT, delta policies
7
431
Insane how good of a simulation @ja_rothschild created with fable 5.2 in the 3 days it has been out!
Rollout on real robot vs same joint targets replayed in digital twin Fable 5.2 built for me. The goal is to do RL and evals for @dm_robots’ actuator unboxing in sim.
I ofc underestimated the challenge. See 🧵 for what I tried and how it went.
1
9
536
Real-Time EXPO-FT is another iteration in the EXPO series, this time tackling policy reactivity on dynamic tasks.
Sampling actions from a VLA like π0.5 takes time. By the time the actions arrive, the observation they were conditioned on is stale. To make the policy more reactive they apply two fixes:
First, the VLA is called asynchronously, ahead of time, conditioned on the actions that will be executed while inference is running (training-time RTC).
Second, the small edit policy is cheap enough to run at the control rate, so it corrects each candidate with the latest observation available after the actions from the base policy arrive.
To speed up training they introduce a separate critic for filtering candidates in noise space during the Bellman backup, so only the best one is denoised. (At test time all candidates are denoised.)
Other interesting things from the paper:
- control frequency 30 Hz, up from 10 Hz in EXPO-FT
- N = 32 candidates instead of 8, each with one edit, so 64 in total
- vision tower of the base policy is not frozen anymore
- no human interventions this time, as far as I can tell
- for some tasks the critics and the edit policy get privileged state information like the ball position and velocity for ball balancing, which I think is kind of a bummer
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning.
The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process.
Additional things I found interesting:
- learner (training and inference) and actor (sensors and motors) run on separate machines
- sparse binary reward is used for the tasks
- for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features
- 10 Hz end-effector velocity control (probably in base frame)
- OpenPI-style augmentation on both camera views
- even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
1
8
469
paper: arxiv.org/abs/2609.18207
code & visualisations: pd-perry.github.io/real-time…
authors: @perryadong @khhung906 @DorsaSadigh @chelseabfinn
99
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning.
The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process.
Additional things I found interesting:
- learner (training and inference) and actor (sensors and motors) run on separate machines
- sparse binary reward is used for the tasks
- for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features
- 10 Hz end-effector velocity control (probably in base frame)
- OpenPI-style augmentation on both camera views
- even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
EXPO is an algorithm to finetune expressive policies with RL.
Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else.
The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data.
The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
paper: arxiv.org/abs/2605.25477
code & visualisations: pd-perry.github.io/expo-ft/
authors: @perryadong @khhung906 @TianGao_19 @DorsaSadigh @chelseabfinn
62
EXPO is an algorithm to finetune expressive policies with RL.
Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else.
The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data.
The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
paper: arxiv.org/abs/2507.07986
code & visualisations: pd-perry.github.io/expo-site…
authors: @perryadong @qiyang_li @DorsaSadigh @chelseabfinn
85
















