Cooperative adaptive optimal output regulation of nonlinear discrete-time multi-agent systems

This paper studies a cooperative adaptive optimal output regulation problem for a class of strict-feedback nonlinear discrete-time (DT) multi-agent systems (MASs) with partially unknown dynamics. Adaptive distributed observer, reinforcement learning (RL) and output regulation techniques are integrated to compute an adaptive near-optimal tracker for each follower. First, under the output regulation theory, the cooperative adaptive optimal output regulation problem is decomposed into a feedforward control design problem which can be addressed by solving nonlinear regulator equations, and an adaptive optimal feedback control problem. Different from general adaptive optimal stabilization problem of nonlinear DT systems, the value function of the optimal output regulation problem is positive semidefinite. This paper analyzes the stability of the closed-loop system without relying on the Lyapunov stability theory and also proposes a policy iteration (PI) approach to approximate the value function with the convergence proof. Then, an adaptive distributed observer is designed for each follower so that they can estimate the state of the leader. A model-state-input structure is developed to find the solutions to regulator equations for each follower and a critic–actor structure is employed to solve the optimal feedback control problem using the measured data based on the neural network (NN) and RL. It is shown that using the estimate values, the tracking errors are uniformly ultimately bounded. A simulation example is given to show the effectiveness of the proposed method.