Adaptive Learning Rate for Follow-the-Regularized-Leader: Competitive Analysis and Best-of-Both-Worlds
arxiv(2024)
摘要
Follow-The-Regularized-Leader (FTRL) is known as an effective and versatile
approach in online learning, where appropriate choice of the learning rate is
crucial for smaller regret. To this end, we formulate the problem of adjusting
FTRL's learning rate as a sequential decision-making problem and introduce the
framework of competitive analysis. We establish a lower bound for the
competitive ratio and propose update rules for learning rate that achieves an
upper bound within a constant factor of this lower bound. Specifically, we
illustrate that the optimal competitive ratio is characterized by the
(approximate) monotonicity of components of the penalty term, showing that a
constant competitive ratio is achievable if the components of the penalty term
form a monotonically non-increasing sequence, and derive a tight competitive
ratio when penalty terms are ξ-approximately monotone non-increasing. Our
proposed update rule, referred to as stability-penalty matching, also
facilitates constructing the Best-Of-Both-Worlds (BOBW) algorithms for
stochastic and adversarial environments. In these environments our result
contributes to achieve tighter regret bound and broaden the applicability of
algorithms for various settings such as multi-armed bandits, graph bandits,
linear bandits, and contextual bandits.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要