The Bootstrap Framework: Generalization Through the Lens of Online Optimization

Preetum Nakkiran
Preetum Nakkiran

international conference on learning representations, 2021.

Cited by: 0|Views17
Weibo:
We show empirical evidence that the performance gap between offline generalization and online optimization is small and propose an alternative framework for studying generalization.

Abstract:

We propose a new framework for reasoning about generalization in deep learning. The core idea is to couple the Real World, where optimizers take stochastic gradient steps on the empirical loss, to an Ideal World, where optimizers take steps on the population loss. This leads to an alternate decomposition of test error into: (1) the Ideal ...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Your rating :
0

 

Tags
Comments