Aurel Arnold (@aurel_arnold) header
Aurel Arnold (@aurel_arnold) profile picture

Building ML models for robotic arms at Dream Machines. Robotics MSc @ETH, prev Serenity @leggedrobotics. Into VLAs, RL.

Zurich, Switzerland
Joined April 2026
Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)Photo by Aurel Arnold (@aurel_arnold)
Aurel Arnold retweeted
We’ve started to record our Paper Club sessions at @dm_robots and post them on YouTube. This is from our last edition a few weeks ago, with a focus on post-training methods for robotic AI, featuring: 1/ Henry Heppe - Real-Time EXPO-FT (@perryadong) 2/ Declan Musto - Scouting the Dynamics Gap (@LisaYishu) 3/ Oscar Devos came from Liechtenstein to present WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL 4/ @aurel_arnold - Reinforcement Learning via Advantage Guidance (@zhiyuan_zhou_) You can find the full recording on Youtube: lnkd.in/ek3bm8Ag Sign up for our next Paper Club on World Models next week Thursday, 27th of November: luma.com/3gu6wflc
4
6
3
69
2,920
I sat down with Bharat to talk about the three bets Dream Machines is built on: > You need to own the customer relationship 
> Self-serve automation is the only thing that scales
> Start with the simplest problem on the factory floor and hill climb
“I don’t think there’s going to be a GPT moment for robotics.” I interviewed Dominique Paul (@DominiqueCAPaul) founder of Dream Machines (@dm_robots) that is building robots to automate work for SME's in Europe After completing his masters from ETH Zurich, and publishing at NeurIPS he freelanced for the next two years to save €50,000 to start a Physical AI company. In the episode we discussed the hidden need of factory owners in Europe (why they can't hire, and how "generalist" robots might be the only solution), how he achieved 98% success using RL, the current landscape of Physical AI companies, how to get paid to collect robotics datasets and more 00:00 Intro 01:24 From ETH & NeurIPS to robotics 04:58 Sleeping in the car, knocking on doors 06:59 What makes factory owners pay 08:48 Labor shortage driving automation 10:06 Journey to first deployment 12:10 Biggest problem with VLAs 13:27 Why no ChatGPT moment for robotics 15:40 Future of robotics data companies 16:51 1.5 hours of data vs 21 hours 19:50 What he looks for in hires 24:36 Benefits of writing online 27:47 Raised in 13 days 30:25 How to get into robotics 35:58 Hardest days 38:54 Build what you’d do for life 40:44 Recommendations for hard days 44:26 What he’s optimizing for 45:00 Scaling operations 47:58 Everything will be obsolete in months
10
6
65
3,406
Check out the first results of policies trained with the custom built UMI device by @gino_kreis!
My first UMI diffusion policies for cube stacking: v1: trained on human-speed demos, collected 1.6x faster than teleop. 91/100 success, and runs 2.4x faster than the teleop policy. v2: trained on demos slowed to teleop speed. 100/100 success. 🧵ACT, delta policies
7
431
Insane how good of a simulation @ja_rothschild created with fable 5.2 in the 3 days it has been out!
Rollout on real robot vs same joint targets replayed in digital twin Fable 5.2 built for me. The goal is to do RL and evals for @dm_robots’ actuator unboxing in sim. I ofc underestimated the challenge. See 🧵 for what I tried and how it went.
1
9
536
watching claude controlling the robot during dinner
4
1
20
1,239
Real-Time EXPO-FT is another iteration in the EXPO series, this time tackling policy reactivity on dynamic tasks. Sampling actions from a VLA like π0.5 takes time. By the time the actions arrive, the observation they were conditioned on is stale. To make the policy more reactive they apply two fixes:
First, the VLA is called asynchronously, ahead of time, conditioned on the actions that will be executed while inference is running (training-time RTC).
Second, the small edit policy is cheap enough to run at the control rate, so it corrects each candidate with the latest observation available after the actions from the base policy arrive. To speed up training they introduce a separate critic for filtering candidates in noise space during the Bellman backup, so only the best one is denoised. (At test time all candidates are denoised.) Other interesting things from the paper: - control frequency 30 Hz, up from 10 Hz in EXPO-FT - N = 32 candidates instead of 8, each with one edit, so 64 in total - vision tower of the base policy is not frozen anymore - no human interventions this time, as far as I can tell - for some tasks the critics and the edit policy get privileged state information like the ball position and velocity for ball balancing, which I think is kind of a bummer
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning. The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process. Additional things I found interesting: - learner (training and inference) and actor (sensors and motors) run on separate machines - sparse binary reward is used for the tasks - for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features - 10 Hz end-effector velocity control (probably in base frame) - OpenPI-style augmentation on both camera views - even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
1
8
469
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning. The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process. Additional things I found interesting: - learner (training and inference) and actor (sensors and motors) run on separate machines - sparse binary reward is used for the tasks - for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features - 10 Hz end-effector velocity control (probably in base frame) - OpenPI-style augmentation on both camera views - even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
EXPO is an algorithm to finetune expressive policies with RL. Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else. The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data. The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
1
1
10
1,355
EXPO is an algorithm to finetune expressive policies with RL. Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else. The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data. The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
1
1
14
1,590
doing a backyard ultra, let’s see…
5
1
16
1,226
last round thats 100km
1
4
63