Identity Crisis: Memorization and Generalization under Extreme Overparameterization
arXiv: Machine Learning, 2019.
We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We examine fully-connected and convolutional networks (FCN and CNN), both linear and nonlinear, initialized randomly and then trained to minimize the reconstructi...More
PPT (Upload PPT)