Sharpness-Aware Optimization Through Variance Suppression on Deep AUC Maximization

Sharpness-aware minimization (SAM) is widely rec-ognized for its ability to improve the generalization of deep neural networks by transforming the optimization problem into a minimax problem, aiming to minimize the maximum loss caused by adversarial parameter perturbations within a neighborhood. However, existing work almost exclusively focuses on the original minimization optimization, with very little attention paid to the minimax optimization. In this paper, we introduce a novel algorithm, VaSSO-SGDAM, by leveraging Variance-Suppressed Sharpness-aware Optimization (VaSSO) for deep AUC maximization. We provide a theoretical convergence analysis of this algorithm, marking it as the first work to achieve such significant theoretical outcomes for this kind of problem. Lastly, we implement our method for optimizing the AUC maximization problem, and the experimental findings validate the efficacy of our approach.