Towards Accurate and Robust Architectures Via Neural Architecture Search

To defend deep neural networks from adversarial attacks, adversarial traininghas been drawing increasing attention for its effectiveness. However, theaccuracy and robustness resulting from the adversarial training are limited bythe architecture, because adversarial training improves accuracy and robustnessby adjusting the weight connection affiliated to the architecture. In thiswork, we propose ARNAS to search for accurate and robust architectures foradversarial training. First we design an accurate and robust search space, inwhich the placement of the cells and the proportional relationship of thefilter numbers are carefully determined. With the design, the architectures canobtain both accuracy and robustness by deploying accurate and robust structuresto their sensitive positions, respectively. Then we propose a differentiablemulti-objective search strategy, performing gradient descent towards directionsthat are beneficial for both natural loss and adversarial loss, thus theaccuracy and robustness can be guaranteed at the same time. We conductcomprehensive experiments in terms of white-box attacks, black-box attacks, andtransferability. Experimental results show that the searched architecture hasthe strongest robustness with the competitive accuracy, and breaks thetraditional idea that NAS-based architectures cannot transfer well to complextasks in robustness scenarios. By analyzing outstanding architectures searched,we also conclude that accurate and robust neural architectures tend to deploydifferent structures near the input and output, which has great practicalsignificance on both hand-crafting and automatically designing of accurate androbust architectures.