Deep Neural Network Learning with Second-Order Optimizers -- a Practical Study with a Stochastic Quasi-Gauss-Newton Method

Thiele Christopher
Thiele Christopher
Cited by: 0|Views12

Abstract:

Training in supervised deep learning is computationally demanding, and the convergence behavior is usually not fully understood. We introduce and study a second-order stochastic quasi-Gauss--Newton (SQGN) optimization method that combines ideas from stochastic quasi-Newton methods, Gauss--Newton methods, and variance reduction to addres...More

Code:

Data:

Full Text
Bibtex
Your rating :
0

 

Tags
Comments