MCE-ERL:Evolutionary Reinforcement Learning Algorithm with Multi-Distribution Critics Evaluation

By combining gradient-based reinforcement learning (RL) with gradient-free evolutionary algorithms (EA), evolutionary reinforcement learning (ERL) algorithms have shown effectiveness in continuous high-dimensional tasks. However, traditional ERL faces challenges such as distributional differences between EA and RL samples affecting critic evaluation, limited exploration due to reward-based metrics, and high interaction costs from low-quality samples. To address these issues, we propose MCE-ERL, an ERL algorithm with multi-distribution critics. We uses a value group and multi-distribution sampling to better cover the sample space, leverages uncertainty for exploration, and applies importance sampling to filter low-quality samples. Experiments on continuous control tasks show that MCE-ERL achieves faster convergence and better performance than baselines, offering insights for reducing sample collection loss in real-world robotic applications.