The Bootstrap Framework: Generalization Through the Lens of Online Optimization

Preetum Nakkiran

Behnam Neyshabur

[0]

Hanie Sedghi

[0]

international conference on learning representations, 2021.

Cited by: 0|Views17

Keywords:

stochastic gradient descentideal worldtest errorpopulation lossdeep learningMore(2+)

Weibo:

We show empirical evidence that the performance gap between offline generalization and online optimization is small and propose an alternative framework for studying generalization.

Abstract:

We propose a new framework for reasoning about generalization in deep learning. The core idea is to couple the Real World, where optimizers take stochastic gradient steps on the empirical loss, to an Ideal World, where optimizers take steps on the population loss. This leads to an alternate decomposition of test error into: (1) the Ideal ...More

Code:

Data:

Full Text

View via Publisher