Designing model-free control law for slowly varying nonlinear systems based on reinforcement learning

A method to control general slowly varying nonlinear systems based on reinforcement learning is proposed. Based on the Q-leaning algorithm a model-free control signal is designed. However, this control signal is numerical and also not smooth enough. Thus a polynomial is fitted to the control signal obtained by Q-learning algorithm. A larger degree of polynomial leads to smaller fitting error. By this procedure, we have a smooth control signal which is easy to implement, moreover, the control signal is in a closed form and is not numerical any longer. This permits deep, elegant and powerful mathematical analysis of the nonlinear system and many properties of the system such as stability, controllability, chaos, limit cycles can be studied. The efficiency of the proposed technique is proved by using it in an example.