What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
NIPS 2020, 2020.
Our work demonstrates that accuracy of a learning algorithm on long tailed data distributions depends on its ability to memorize the labels
Deep learning algorithms are well-known to have a propensity for fitting the training data very well and often fit even outliers and mislabeled data points. Such fitting requires memorization of training data labels, a phenomenon that has attracted significant research interest but has not been given a compelling explanation so far. A r...More
PPT (Upload PPT)