Performance Optimization and Fault-Tolerance of Highly Dynamic Systems via Q-Learning With an Incrementally Attached Controller Gain System

High-performance and reliable control of systems that are highly dynamic and open-loop unstable is challenging but of considerable practical interest. Thus, this article investigates the performance optimization and fault tolerance of highly dynamic systems. First, an incremental control structure is proposed, where a controller gain system is attached to the predesigned controller, and by reconfiguring the controller gain system, the performance can be equivalently optimized as configuring the predesigned one. The incremental attachment of the controller gain system does not modify the existing control system, and it can be easily attached via various communication channels. Second, a structure integrating fault-tolerance strategy and hardware redundancy is proposed. Under this structure, command fusion and fault-tolerance strategies are developed where the control commands from different control units are optimally fused, and each control unit can be reconfigured w.r.t. the performance of the other ones. Furthermore, Q-learning algorithms are developed to realize the proposed structures and strategies in real-time model-freely. As such, varying operational conditions of the highly dynamic system can be tackled. Finally, the proposed structures and algorithms are validated case by case to show their effectiveness.