World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays

Going beyond predicting robot actions, World Action Models (WAMs) can also generate future visual observations. We build on this generative capability to propose Recurrent Generative Replay (REGEN), a continual imitation learning framework that synthesizes pseudo-replay trajectories, enabling a robot policy to rehearse previously learned tasks without storing their original human demonstrations. During continual adaptation, REGEN recursively queries the WAM to synthesize pseudo-replay trajectories conditioned only on prior task instructions and current-task observations. Experiments in both simulation and real-world manipulation settings show that REGEN reduces catastrophic forgetting by up to 50% relative to sequential fine-tuning, while approaching the performance of privileged experience replay methods that require access to real replay data. Finally, we analyze the factors limiting generated replay, identifying long-horizon visual degradation and action-observation inconsistency as the primary bottlenecks. Our results establish WAMs as a promising foundation for continual robot learning without stored demonstrations.

REGEN operates by leveraging a pretrained World Action Model (WAM) as its own replay mechanism. When adapting to a new task, REGEN generates pseudo-demonstrations for each previous task by conditioning the WAM on the prior task's language instruction and initializing the rollout from a real observation sampled from the current task's demonstrations.

Generation proceeds in three phases. First, an initialization phase seeds the rollout with one real action chunk from the current task. Second, a recurrent generation phase takes over: the model's own predicted future observations are fed back as inputs, producing a fully synthetic trajectory without any stored data. Generation terminates either at a maximum horizon or when the WAM's reward head predicts task completion with high confidence.

The resulting pseudo-trajectories are combined with the current task's demonstrations and used to fine-tune the policy via behavioral cloning. This allows the WAM to simultaneously acquire new skills and rehearse previously learned ones, using only its own generative outputs as memory.

We evaluate REGEN on three LIBERO benchmark suites against Sequential Fine-Tuning (Seq-FT), Experience Replay (ER, upper bound), and Rollouts-as-Replay (RAR). ER is treated as a privileged reference since it assumes access to stored demonstrations. Higher FWT and AUC are better; lower NBT is better.

We compare REGEN against π0.5, a state-of-the-art VLA pretrained on large-scale robotic data, on LIBERO-Goal. Despite π0.5's significantly larger pretraining dataset, catastrophic forgetting persists. REGEN achieves stronger retention by leveraging WAM's joint action-observation generative modeling.

We evaluate REGEN on a real xArm7 manipulator across three sequential pick-and-place tasks: Put carrot in bowl (T1), Put carrot on plate (T2), and Put eggplant in bowl (T3). Each policy is evaluated over 10 randomized trials per task.

Demonstration videos of REGEN in simulation and real-world settings coming soon.