Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defense
arxiv(2024)
摘要
Recent studies have revealed the vulnerability of Deep Neural Networks (DNNs)
to adversarial examples, which can easily fool DNNs into making incorrect
predictions. To mitigate this deficiency, we propose a novel adversarial
defense method called "Immunity" (Innovative MoE with MUtual information &
positioN stabilITY) based on a modified Mixture-of-Experts (MoE) architecture
in this work. The key enhancements to the standard MoE are two-fold: 1)
integrating of Random Switch Gates (RSGs) to obtain diverse network structures
via random permutation of RSG parameters at evaluation time, despite of RSGs
being determined after one-time training; 2) devising innovative Mutual
Information (MI)-based and Position Stability-based loss functions by
capitalizing on Grad-CAM's explanatory power to increase the diversity and the
causality of expert networks. Notably, our MI-based loss operates directly on
the heatmaps, thereby inducing subtler negative impacts on the classification
performance when compared to other losses of the same type, theoretically.
Extensive evaluation validates the efficacy of the proposed approach in
improving adversarial robustness against a wide range of attacks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要