Limitations of the empirical Fisher approximation for natural gradient descent

Frederik Kunstner
Frederik Kunstner
Lukas Balles
Lukas Balles

NeurIPS, pp. 4158-4169, 2019.

Cited by: 2|Views19
EI

Abstract:

Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have advocated an approximation known as the empirical Fisher, drawing connections between approximate second-ord...More

Code:

Data:

Full Text
Bibtex
Your rating :
0

 

Tags
Comments