# Lipschitz and Comparator-Norm Adaptivity in Online Learning

COLT, pp.2858-2887, (2020)

EI

Weibo:

Abstract:

We study Online Convex Optimization in the unbounded setting where neither predictions nor gradient are constrained. The goal is to simultaneously adapt to both the sequence of gradients and the comparator. We first develop parameter-free and scale-free algorithms for a simplified setting with hints. We present two versions: the first a...More

Code:

Data:

ZH

Introduction

- The authors consider the setting of online convex optimization where the goal is to make sequential predictions to minimize a certain notion of regret.
- At the beginning of each round t ě 1, a learner predicts wpt in some convex set W Ď Rd in dimension d P N.
- The authors are interested in online algorithms which can guarantee a good regret bound against any comparator vector w P W, even when W is unbounded, and without prior knowledge of the maximum norm L :“ maxtďT }gt} of the observed sub-gradients.
- The authors refer to L as the Lipschitz constant

Highlights

- We consider the setting of online convex optimization where the goal is to make sequential predictions to minimize a certain notion of regret
- We show how our algorithms can be applied to learn linear models
- As we saw in Corollary 3, given a base algorithm A, which takes a sequence of hints phtq such that ht ě Lt, for all t ě 1, and suffers regret RTApwq against comparator w P W, there exists an algorithm B for the setting without hints which suffers the same regret against w up to an additive pptheeanntaailltttyyisLLnToTt}}wpwo}}s33siibsleLnotTotairmempplaarocxvetPatrbhTlesepBieftn,oawnltehyeiLnresTiBsatst mo“nařxatPrtser“Tg1sr}eBgt tsb}bo{yuLntthd.eIotnfytpohircidsaelsrleyOcrstipmo?naT,llweqr.eqWsuheaonawtlisttyoha?sthVtohTwe, where
- As in Section 3, we focus on algorithms which make predictions based on observed sub-gradients; in this case, gt “ xtδt P xt Bp0,1q pyt, yptq “ Bftpuptq, t ě 1, where ftpwq “ pyt, w xtq

Results

- The authors improve on current state-of-the-art algorithms in two ways; First, the authors provide a scale-invariant algorithm which guarantees regret bound, against any w P Rd, of order db ÿ |wi|.

Conclusion

- The algorithms designed can be used in the role of algorithm A in the reductions presented in Section 2.2.
- This will yield algorithms which achieve the goal; they adapt to the norm of the comparator and the Lipschitz constant and are completely scale-free, for both bounded and unbounded sets, without requiring hints.
- By restricting gt to be aligned with wpt, the problem reduces to finding x “ }gt} such that x}wpt} ́ |}wpt} ̈ pLt1 _ xq{2 ́ St1| ě 0

Summary

## Introduction:

The authors consider the setting of online convex optimization where the goal is to make sequential predictions to minimize a certain notion of regret.- At the beginning of each round t ě 1, a learner predicts wpt in some convex set W Ď Rd in dimension d P N.
- The authors are interested in online algorithms which can guarantee a good regret bound against any comparator vector w P W, even when W is unbounded, and without prior knowledge of the maximum norm L :“ maxtďT }gt} of the observed sub-gradients.
- The authors refer to L as the Lipschitz constant
## Results:

The authors improve on current state-of-the-art algorithms in two ways; First, the authors provide a scale-invariant algorithm which guarantees regret bound, against any w P Rd, of order db ÿ |wi|.## Conclusion:

The algorithms designed can be used in the role of algorithm A in the reductions presented in Section 2.2.- This will yield algorithms which achieve the goal; they adapt to the norm of the comparator and the Lipschitz constant and are completely scale-free, for both bounded and unbounded sets, without requiring hints.
- By restricting gt to be aligned with wpt, the problem reduces to finding x “ }gt} such that x}wpt} ́ |}wpt} ̈ pLt1 _ xq{2 ́ St1| ě 0

Related work

**Related Work For an overview of Online**

Convex Optimization in the bounded setting, we refer to the textbook (Hazan, 2016). The unconstrained case was first studied by McMahan and Streeter (2010). A powerful methodology for the unbounded case is Coin Betting by Orabona and Pal (2016a). Even though not always visible, our potential functions are inspired by this style of thinking. We build our unbounded OCO learner by targeting a specific other constrained problem. We further employ several general reductions from the literature, including gradient clipping Cutkosky (2019), the constrained-to-unconstrained reduction Cutkosky and Orabona (2018), and the restart wrapper to pacify the final-vs-initial scale ratio appearing inside logarithms by Mhammedi et al (2019). Our analysis is, at its core, proving a certain minimax result about sufficient-statisticbased potentials reminiscent of the Burkholder approach pioneered by Foster et al (2017, 2018). Applications for scale-invariant learning in linear models were studied by Kempka et al (2019). For our multidimensional learner we took inspiration from the Gaussian Exp-concavity step in the analysis of the MetaGrad algorithm by Van Erven and Koolen (2016).

Reference

- Nicolo Cesa-Bianchi, Yoav Freund, David Haussler, David P. Helmbold, Robert E. Schapire, and Manfred K. Warmuth. How to use expert advice. J. ACM, 44(3):427–485, 1997.
- Ashok Cutkosky. Artificial constraints and hints for unbounded online learning. In Conference on Learning Theory, pages 874–894, 2019.
- Ashok Cutkosky and Kwabena Boahen. Online learning without prior information. In Conference on Learning Theory, pages 643–677, 2017.
- Ashok Cutkosky and Francesco Orabona. Black-box reductions for parameter-free online learning in Banach spaces. In Conference On Learning Theory, COLT 2018, Stockholm, Sweden, 6-9 July 2018., pages 1493–1529, 2018.
- Tim van Erven and Wouter M. Koolen. Metagrad: Multiple learning rates in online learning. In Advances in Neural Information Processing Systems 29 (NIPS), pages 3666–3674, 2016.
- Dylan J. Foster, Alexander Rakhlin, and Karthik Sridharan. Zigzag: A new approach to adaptive online learning. In Proc. of the 2017 Annual Conference on Learning Theory (COLT), pages 876–924, 2017.
- Dylan J. Foster, Alexander Rakhlin, and Karthik Sridharan. Online learning: Sufficient statistics and the Burkholder method. In Conference On Learning Theory, COLT 2018, Stockholm, Sweden, 6-9 July 2018., pages 3028–3064, 2018.
- Elad Hazan. Introduction to online convex optimization. Foundations and Trends in Optimization, 2 (3–4):157–325, 2016.
- Jean-Baptiste Hiriart-Urruty and Claude Lemarechal. Fundamentals of Convex Analysis. Springer Science & Business Media, 2004.
- Michal Kempka, Wojciech Kotłowski, and Manfred K. Warmuth. Adaptive scale-invariant online algorithms for learning linear models. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pages 3321–3330, 2019.
- Wojciech Kotłowski. Scale-invariant unconstrained online learning. In International Conference on Algorithmic Learning Theory, ALT 2017, 15-17 October 2017, Kyoto University, Kyoto, Japan, pages 412–433, 2017.
- H. Brendan McMahan and Francesco Orabona. Unconstrained online linear learning in Hilbert spaces: Minimax algorithms and normal approximations. In Conference on Learning Theory, pages 1020–1039, 2014.
- H. Brendan McMahan and Matthew J. Streeter. Adaptive bound optimization for online convex optimization. In Proceedings of the 23rd Annual Conference on Learning Theory (COLT), pages 244–256, 2010.
- LIPSCHITZ-COMPARATOR-NORM-ADAPTIVITY Zakaria Mhammedi, Wouter M. Koolen, and Tim van Erven. Lipschitz adaptivity with multiple learning rates in online learning. In Conference on Learning Theory, COLT 2019, 25-28 June 2019, Phoenix, AZ, USA, pages 2490–2511, 2019. Francesco Orabona and David Pal. Coin betting and parameter-free online learning. In Advances in Neural Information Processing Systems, pages 577–585, 2016a. Francesco Orabona and David Pal. Open problem: Parameter-free and scale-free online algorithms. In Conference on Learning Theory, pages 1659–1664, 2016b. Stephane Ross, Paul Mineiro, and John Langford. Normalized online learning. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 537–545, 2013. Olivier Wintenberger. Optimal learning with Bernstein online aggregation. Machine Learning, 106 (1):119–141, 2017.

Tags

Comments