Aggregated Momentum: Stability Through Passive Damping

ICLR, 2019.

Cited by: 29|Views82
EI

Abstract:

Momentum is a simple and widely used trick which allows gradient-based optimizers to pick up speed along low curvature directions. Its performance depends crucially on a damping coefficient $\beta$. Large $\beta$ values can potentially deliver much larger speedups, but are prone to oscillations and instability; hence one typically resor...More

Code:

Data:

Full Text
Bibtex