Reinforcement Learning-Assisted Composite Adaptive Control for Time-Varying Parameters

Abstract Adaptive control methods have received a lot of interest to control uncertain systems with parametric uncertainties. In particular, composite adaptation law that incorporates a memory storing the past trajectory data is promising, because it has an exponential convergent rate for both the tracking error and the parameter estimation under a mild condition of excitation. In this study, this research direction is extended to cope with uncertain parameters that change over time, which is difficult to solve with traditional memory-based methods. The problem is formulated into a Markov decision process, and a reinforcement learning algorithm is adopted to solve the optimal decision making problem. The proposed formulation preserves the stability of the original composite adaptive system, and the reinforcement learning agent can learn the optimal composite strategy.