Data Drift in DL: Lessons Learned from Encrypted Traffic Classification

Deep learning models have shown to achieve high performance in encrypted traffic classification. However, when it comes to production use, multiple factors challenge the performance of these models. The emergence of new network traffic protocols, especially at the application-layer, as well as updates to previous protocols affect the patterns in input data, making the model's previously learned patterns obsolete. Furthermore, proposed model architectures are usually tested on datasets collected in controlled settings, which makes the reported performances unreliable for production use. In this paper, we study how the performances of two high-performing traffic classifiers change on multiple real-world datasets collected over the course of two years. We investigate the changes in traffic data patterns showing the extent to which these changes reduce the performance of the two models. Furthermore, we propose architectural adaptations to a flow time-series based traffic classifier, showing that they improve accuracy by 4.8%.