Mitigating Covariate Shift in Offline Imitation Learning Via Trajectory Synthesis

Offline imitation learning (IL) is a practical route to embodied policy learning when online exploration is unsafe, expensive, or unavailable. Its main limitation is the closed-loop covariate shift: a policy trained only on fixed demonstrations can enter states outside the expert support, where behavioral cloning provides no recovery supervision and small errors compound. We propose Synthetic Imitation Learning (SynIL), an offline IL framework that augments demonstrations with synthetic recovery trajectories rather than only filtering or reweighting logged data. SynIL uses generative dynamics models to enlarge the training distribution around failure-prone states and retains transitions whose successor states move back toward the expert state manifold. In low-dimensional benchmarks, SynIL trains a lightweight latent world model to imagine short-horizon rollouts from suboptimal states and uses a positive-unlabeled state discriminator to assign proxy rewards without access to task rewards. In high-dimensional robot manipulation, SynIL instantiates the same principle with a training-free visual synthesis pipeline that combines pretrained video generation, segmentation, point tracking, and monocular depth estimation to recover imageaction trajectories. Experiments on 15 D4RL tasks show that SynIL achieves first- or second-place performance on 8 tasks across navigation, locomotion, and manipulation. Real-world UR5 experiments improve the average success rate by 17.5% over behavior cloning, indicating that synthetic recovery data can improve robustness without online environment interaction.