Model-free Optimal Consensus Control for Multi-Agent Systems Via DDPG-based Event-Triggered Adaptive Dynamic Programming Method

This paper primarily addresses the design of distributed optimal cooperative controllers and the utilization of a reinforcement learning (RL)-based event-triggered mechanism for multi-agent systems (MASs) with unknown dynamics. By setting an extra compensator, the augmented system is constructed to overcome the dependence for system dynamics. Then, to address the issue of computational burden, we utilize an event-triggered mechanism based on reinforcement learning (RL) and neural networks (NNs) to implement the adaptive dynamic programming (ADP) algorithm. Additionally, we take into consideration the trade-off between computational burden and achieving consensus control by introducing a weighting factor in the reward design for MASs. With this reward design, we present an algorithm based on the deep deterministic policy gradient (DDPG) algorithm to learn the event-triggered condition for MASs and achieve a balance between these two factors. The event-triggered mechanism of our algorithm can also identify constraints such as time limitations or computational resource restrictions, aiming to achieve consensus control without violating these constraints. We demonstrate the absence of Zeno behavior and the uniform ultimate boundedness (UUB) of both local consensus error and weight estimation error. Finally, simulation results illustrate the effectiveness of the control algorithm and the weighting factor.