Decision Transformer as a Foundation Model for Partially Observable Continuous Control
arxiv(2024)
摘要
Closed-loop control of nonlinear dynamical systems with partial-state
observability demands expert knowledge of a diverse, less standardized set of
theoretical tools. Moreover, it requires a delicate integration of controller
and estimator designs to achieve the desired system behavior. To establish a
general controller synthesis framework, we explore the Decision Transformer
(DT) architecture. Specifically, we first frame the control task as predicting
the current optimal action based on past observations, actions, and rewards,
eliminating the need for a separate estimator design. Then, we leverage the
pre-trained language models, i.e., the Generative Pre-trained Transformer (GPT)
series, to initialize DT and subsequently train it for control tasks using
low-rank adaptation (LoRA). Our comprehensive experiments across five distinct
control tasks, ranging from maneuvering aerospace systems to controlling
partial differential equations (PDEs), demonstrate DT's capability to capture
the parameter-agnostic structures intrinsic to control tasks. DT exhibits
remarkable zero-shot generalization abilities for completely new tasks and
rapidly surpasses expert performance levels with a minimal amount of
demonstration data. These findings highlight the potential of DT as a
foundational controller for general control applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要