Fault Tolerant Controller for Hexacopter Using Reinforcement Learning and Extended State Observer

In this study, a fault tolerant controller is designed for a hexacopter with actuator faults using reinforcement learning method and extended state observer. The control system consists of an outer-loop controller, which is designed to track the horizontal position using a reinforcement learning scheme, and an inner-loop controller, which is designed to track the altitude and attitude using extended state observer and the baseline controller based on nonlinear dynamic inversion. Using integral reinforcement learning based on value iteration, the optimal control problem is solved for a partially known system model, with an assumption initial stabilizing control is not required. The disturbances due to the actuator faults are estimated by an extended state observer so that the generalized forces, the virtual controller, may be designed. An optimization-based constrained control allocation method is adopted to allocate the desired generalized forces to six rotors considering the saturation of the actuators. Numerical simulation is performed to demonstrate the effectiveness of the proposed controller for a 6-Degrees of Freedom hexacopter system.