Closed-loop Feedback Optimization for Autonomous Vehicles Using Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) has demonstrated significant potential in tackling the complex challenges of autonomous vehicles, particularly in decision-making and motion planning. However, many existing DRL methods optimize individual components in isolation, leading to a lack of integration across modules. Additionally, current reinforcement learning methods often focus on high-reward samples while neglecting the valuable information contained in low-reward samples. Furthermore, the efficiency of action search requires further improvement. To overcome these challenges, we propose a novel closed-loop DRL framework that utilizes feedback to simultaneously optimize both decision-making and motion planning. To address these, we propose a novel closed-loop DRL framework that leverages closed-loop feedback to optimize both decision-making and motion planning concurrently. The feedback framework refines the DRL process by utilizing the deviation between real and predicted trajectories, enabling continuous improvement of the system’s decision-making capabilities. The framework also incorporates a dual-value priority sampling strategy, which enhances the utilization of low-reward samples by prioritizing the sampling of high-value samples across categories. Additionally, the framework integrates a hybrid action selection mechanism that combines rule-based and learning-based mechanism, effectively reducing exploration inefficiencies in the early stages of DRL while maintaining long-term optimization through experience-driven learning. Experimental results demonstrate that the proposed framework outperforms state-of-the-art models, showing strong potential for real-world applications.