Improving Generalization and Robustness in Offline Reinforcement Learning Via Boundary-Aware Data Augmentation

Current offline reinforcement learning (ORL) algorithms tend to overfit the training dataset and exhibit poor in-distribution generalization and robustness performance when deployed to real environments, thus compromising their effectiveness. Existing methods typically enhance in-distribution generalization and robustness by leveraging regularization techniques widely used in computer vision. However, due to the high sensitivity of low-level physical signals to distributional shifts, these methods still suffer from notable limitations in in-distribution generalization and robustness, making it difficult to achieve stable performance in complex environments. To address this issue, we theoretically analyze the error bounds of the transition function under random episode interpolation, showing that the interpolation error increases with the distance between states. Based on this insight, we propose a method called $\bf{B}$oundary-$\bf{A}$ware $\bf{D}$ata $\bf{A}$ugmentation (BADA), which leverages neighboring states to construct interpolation boundaries, enabling the generation of synthetic data that more faithfully preserves the original data distribution. We first conduct qualitative studies in a toy environment, showing that BADA generates mixed samples that preserve desirable policy smoothness while faithfully maintaining multimodal value distributions. Extensive experiments on limited offline datasets demonstrate that BADA attains state-of-the-art performance across diverse benchmarks. We further evaluate BADA under noise contamination and both white-box and black-box adversarial attacks, with results demonstrating that the proposed method enhances robustness to state perturbations. Finally, we investigate offline datasets with noisy transition dynamics and find that BADA effectively mitigates performance degradation and improves robustness against transition model perturbations.