The Local Elasticity of Neural Networks
Zhang et al observe that neural networks can perfectly fit corrupted labels while maintaining a certain amount of generalization power1. We complement this line of findings by proposing a hypothesis that fundamentally distinguishes neural networks from linear classifiers2. This h...
This paper presents a phenomenon in neural networks that we refer to as local elasticity. Roughly speaking, a classifier is said to be locally elastic if its prediction at a feature vector x' is not significantly perturbed, after the classifier is updated via stochastic gradient descent at a (labeled) feature vector x that is dissimilar t...More
PPT (Upload PPT)