An Invitation to Deep Reinforcement Learning
CoRR(2023)
摘要
Training a deep neural network to maximize a target objective has become the
standard recipe for successful machine learning over the last decade. These
networks can be optimized with supervised learning, if the target objective is
differentiable. For many interesting problems, this is however not the case.
Common objectives like intersection over union (IoU), bilingual evaluation
understudy (BLEU) score or rewards cannot be optimized with supervised
learning. A common workaround is to define differentiable surrogate losses,
leading to suboptimal solutions with respect to the actual objective.
Reinforcement learning (RL) has emerged as a promising alternative for
optimizing deep neural networks to maximize non-differentiable objectives in
recent years. Examples include aligning large language models via human
feedback, code generation, object detection or control problems. This makes RL
techniques relevant to the larger machine learning audience. The subject is,
however, time intensive to approach due to the large range of methods, as well
as the often very theoretical presentation. In this introduction, we take an
alternative approach, different from classic reinforcement learning textbooks.
Rather than focusing on tabular problems, we introduce reinforcement learning
as a generalization of supervised learning, which we first apply to
non-differentiable objectives and later to temporal problems. Assuming only
basic knowledge of supervised learning, the reader will be able to understand
state-of-the-art deep RL algorithms like proximal policy optimization (PPO)
after reading this tutorial.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要