Technical Report: Adaptive Control for Linearizable Systems Using On-Policy Reinforcement Learning
arxiv(2020)
摘要
This paper proposes a framework for adaptively learning a feedback linearization-based tracking controller for an unknown system using discrete-time model-free policy-gradient parameter update rules. The primary advantage of the scheme over standard model-reference adaptive control techniques is that it does not require the learned inverse model to be invertible at all instances of time. This enables the use of general function approximators to approximate the linearizing controller for the system without having to worry about singularities. However, the discrete-time and stochastic nature of these algorithms precludes the direct application of standard machinery from the adaptive control literature to provide deterministic stability proofs for the system. Nevertheless, we leverage these techniques alongside tools from the stochastic approximation literature to demonstrate that with high probability the tracking and parameter errors concentrate near zero when a certain persistence of excitation condition is satisfied. A simulated example of a double pendulum demonstrates the utility of the proposed theory. 1
更多查看译文
关键词
discrete-time model-free policy-gradient parameter update rules,inverse model,function approximators,learning system,linearizable systems,on-policy reinforcement learning,feedback linearization-based tracking,model-reference adaptive control techniques,excitation condition,machine learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络