Sharpness-Aware Minimization for Efficiently Improving Generalization

Pierre Foret
Pierre Foret
Ariel Kleiner
Ariel Kleiner

international conference on learning representations, 2020.

Cited by: 0|Views17
Weibo:
Motivated by the connection between geometry of the loss landscape and generalization, we introduce a procedure for simultaneously minimizing loss value and loss sharpness.

Abstract:

In today's heavily overparameterized models, the value of the training loss provides few guarantees on model generalization ability. Indeed, optimizing only the training loss value, as is commonly done, can easily lead to suboptimal model quality. Motivated by the connection between geometry of the loss landscape and generalization---in...More
0
Full Text
Bibtex
Weibo
Your rating :
0

 

Tags
Comments