Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
CoRR(2024)
摘要
Deep Reinforcement Learning (DRL) techniques have received significant
attention in control and decision-making algorithms. Most applications involve
complex decision-making systems, justified by the algorithms' computational
power and cost. While model-based versions are emerging, model-free DRL
approaches are intriguing for their independence from models, yet they remain
relatively less explored in terms of performance, particularly in applied
control. This study conducts a thorough performance analysis comparing the
data-driven DRL paradigm with a classical state feedback controller, both
designed based on the same cost (reward) function of the linear quadratic
regulator (LQR) problem. Twelve additional performance criteria are introduced
to assess the controllers' performance, independent of the LQR problem for
which they are designed. Two Deep Deterministic Policy Gradient (DDPG)-based
controllers are developed, leveraging DDPG's widespread reputation. These
controllers are aimed at addressing a challenging setpoint tracking problem in
a Non-Minimum Phase (NMP) system. The performance and robustness of the
controllers are assessed in the presence of operational challenges, including
disturbance, noise, initial conditions, and model uncertainties. The findings
suggest that the DDPG controller demonstrates promising behavior under rigorous
test conditions. Nevertheless, further improvements are necessary for the DDPG
controller to outperform classical methods in all criteria. While DRL
algorithms may excel in complex environments owing to the flexibility in the
reward function definition, this paper offers practical insights and a
comparison framework specifically designed to evaluate these algorithms within
the context of control engineering.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要