Junbang Liang

With video generation as a proxy for robot policy learning, Video Policy achieves the highest overall performance in the Libero10 benchmark compared to strong baselines.

We analyze 2-stage training compared to jointly training the video model and the action decoder. Notably, 2-stage training achieves significantly higher success rates, suggesting that video generation serves as a more general learning objective than action prediction.

We show generalization to unseen tasks using action-free video data. While both our action head and baseline DP train on 12 tasks (left), our video model trains on action-free videos from 12 tasks (right) and performs the tasks without action supervision training!

Video Policy takes an initial image and future noise vectors as input, then jointly denoise video frames and actions. The intermediate video features are used to condition the action decoder, enabling modular training and action-free learning through video generation alone.