PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor
CVPR 2024(2024)
摘要
Adversarial robustness of the neural network is a significant concern when it
is applied to security-critical domains. In this situation, adversarial
distillation is a promising option which aims to distill the robustness of the
teacher network to improve the robustness of a small student network. Previous
works pretrain the teacher network to make it robust to the adversarial
examples aimed at itself. However, the adversarial examples are dependent on
the parameters of the target network. The fixed teacher network inevitably
degrades its robustness against the unseen transferred adversarial examples
which targets the parameters of the student network in the adversarial
distillation process. We propose PeerAiD to make a peer network learn the
adversarial examples of the student network instead of adversarial examples
aimed at itself. PeerAiD is an adversarial distillation that trains the peer
network and the student network simultaneously in order to make the peer
network specialized for defending the student network. We observe that such
peer networks surpass the robustness of pretrained robust teacher network
against student-attacked adversarial samples. With this peer network and
adversarial distillation, PeerAiD achieves significantly higher robustness of
the student network with AutoAttack (AA) accuracy up to 1.66
natural accuracy of the student network up to 4.72
TinyImageNet dataset.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要