Asynchronous Multi-agent Pareto Optimization for Diverse UAV Maneuver Strategy Generation

Recent advances have witnessed that Multi-Agent Reinforcement Learning (MARL) makes significant progress in Multi-UAV maneuver strategy generation. Difference from traditional MARL tasks, Multi-UAV combat scenarios are always in high dynamism and complexity, and exploring varying available maneuver strategies is necessary. In this paper, to extend the diverse maneuver strategy, we formalize the problem as the multi-objective optimization problem and propose an asynchronous Pareto-based multi-agent population optimization method. Besides, we propose the tolerance method to alleviate the Pareto front shock problem in the asynchronous Pareto optimization process. Finally, a 2V2 6-DOF UAV simulation environment is designed to evaluate the performance of the proposed methods. Experimental results show that our method can efficiently learn multiple maneuver strategies, such as counterattack and defense penetration.