Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximation
CoRR(2024)
摘要
In the realm of reinforcement learning (RL), accounting for risk is crucial
for making decisions under uncertainty, particularly in applications where
safety and reliability are paramount. In this paper, we introduce a general
framework on Risk-Sensitive Distributional Reinforcement Learning (RS-DisRL),
with static Lipschitz Risk Measures (LRM) and general function approximation.
Our framework covers a broad class of risk-sensitive RL, and facilitates
analysis of the impact of estimation functions on the effectiveness of RSRL
strategies and evaluation of their sample complexity. We design two innovative
meta-algorithms: , a model-based strategy for model-based
function approximation, and , a model-free approach for
general value function approximation. With our novel estimation techniques via
Least Squares Regression (LSR) and Maximum Likelihood Estimation (MLE) in
distributional RL with augmented Markov Decision Process (MDP), we derive the
first 𝒪(√(K)) dependency of the regret upper bound
for RSRL with static LRM, marking a pioneering contribution towards
statistically efficient algorithms in this domain.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要