On Empirical Comparisons of Optimizers for Deep Learning

Shallue Christopher J.
Shallue Christopher J.
Nado Zachary
Nado Zachary
Cited by: 44|Views40

Abstract:

Selecting an optimizer is a central step in the contemporary deep learning pipeline. In this paper, we demonstrate the sensitivity of optimizer comparisons to the metaparameter tuning protocol. Our findings suggest that the metaparameter search space may be the single most important factor explaining the rankings obtained by recent empi...More

Code:

Data:

Full Text
Bibtex
Your rating :
0

 

Tags
Comments