Limitations of the Empirical Fisher Approximation for Natural Gradient Descent

Frederik Kunstner
Frederik Kunstner
Lukas Balles
Lukas Balles

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019.

Cited by: 0|Views1

Abstract:

Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between approximate second-ord...More

Code:

Data:

Get fulltext within 24h
Bibtex
Your rating :
0

 

Tags
Comments