Entropy-SGD: biasing gradient descent into wide valleys
Journal of Statistical Mechanics: Theory and Experiment, pp. 1240182019.
This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenvalues. We leverage u...More
Full Text (Upload PDF)
PPT (Upload PPT)