Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), pp. 8080-8091, 2019.

Cited by: 17|Views80
EI

Abstract:

Natural gradient descent has proven effective at mitigating the effects of pathological curvature in neural network optimization, but little is known theoretically about its convergence properties, especially for nonlinear networks. In this work, we analyze for the first time the speed of convergence of natural gradient descent on nonline...More

Code:

Data:

Your rating :
0

 

Tags
Comments