Understanding deep learning requires rethinking generalization
ICLR, Volume abs/1611.03530, 2017.
These models are in principle rich enough to memorize the training data. This situation poses a conceptual challenge to statistical learning theory as traditional measures of model complexity struggle to explain the generalization ability of large artificial neural networks
Despite their massive size, successful deep artificial neural networks canexhibit a remarkably small difference between training and test performance.Conventional wisdom attributes small generalization error either to propertiesof the model family, or to the regularization techniques used during training.Through extensive systematic exper...More
PPT (Upload PPT)