UAD-DPN: An Unknown Attack Detection Method for Encrypted Traffic Based on Deep Prototype Network

Abstract Intrusion detection systems (IDS) are well-known means of quickly detecting attacks, which can effectively detect known attacks available during training. However, when the system operates in a real open network environment, the attacks which it experiences may differ from those learned during training, which we call unknown attacks. Unknown attacks are significant threats, and their effects are the same as zero days. The main challenge of IDS is to detect unknown attacks and distinguish them from benign traffic and existing known attacks. There-fore, it is very importance to quantify to what extent an IDS can detect unknown attacks. But most existing deep learning methods for unknown attack detection cannot clearly recognize the deep features of unknown attack classes, which are inherently inaccurate. To solve these problems, an innovative unknown attack detection approach based on deep prototype network (UAD-DPN) is proposed to enhance the accuracy and efficiency of encrypted unknown attack detection. First, we employ an encrypted traffic spatiotemporal fusion feature extraction network to improve the feature representation ability. Then, we propose an innovative prototype-based encrypted traffic feature space learning model, which uses discriminative loss and open loss training models to improve the performance of encrypted unknown attacks detection. Finally, an unknown attack identification method based on the nearest prototype rule and a three-stage training approach for UAD-DPN model are designed to conveniently and effectively identify known attacks and reject unknown attacks. The experimental results demonstrated that the proposed UAD-DPN is very effective to detect both known and unknown attacks for encrypted traffic with higher accuracy and efficiency. Meanwhile, UAD-DPN have good application prospects in network intrusion detection system under the complex open network environment.