On the Duality Between Sharpness-Aware Minimization and Adversarial Training
CoRR(2024)
摘要
Adversarial Training (AT), which adversarially perturb the input samples
during training, has been acknowledged as one of the most effective defenses
against adversarial attacks, yet suffers from a fundamental tradeoff that
inevitably decreases clean accuracy. Instead of perturbing the samples,
Sharpness-Aware Minimization (SAM) perturbs the model weights during training
to find a more flat loss landscape and improve generalization. However, as SAM
is designed for better clean accuracy, its effectiveness in enhancing
adversarial robustness remains unexplored. In this work, considering the
duality between SAM and AT, we investigate the adversarial robustness derived
from SAM. Intriguingly, we find that using SAM alone can improve adversarial
robustness. To understand this unexpected property of SAM, we first provide
empirical and theoretical insights into how SAM can implicitly learn more
robust features, and conduct comprehensive experiments to show that SAM can
improve adversarial robustness notably without sacrificing any clean accuracy,
shedding light on the potential of SAM to be a substitute for AT when accuracy
comes at a higher priority. Code is available at
https://github.com/weizeming/SAM_AT.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要