Extreme Memorization via Scale of Initialization

international conference on learning representations, 2021.

Cited by: 0|Views12
Weibo:
Through studying the effect of scale of initialization on generalization, we come up with an alignment measure that correlations with generalization of deep models.

Abstract:

We construct an experimental setup in which changing the scale of initialization strongly impacts the implicit regularization induced by SGD, interpolating from good generalization performance to completely memorizing the training set while making little progress on the test set. Moreover, we find that the extent and manner in which gener...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Your rating :
0

 

Tags
Comments