DeepGuard: Efficient Anomaly Detection in SDN With Fine-Grained Traffic Flow Monitoring

Software-Defined Networking (SDN) leverages the implementation of reliable, flexible and efficient network security mechanisms which make use of novel techniques such as artificial intelligence (AI) and machine learning (ML). In particular, these techniques - together with SDN - are the key enablers for the design of anomaly detection methods which are based on efficient traffic flow monitoring. In this paper, we tackle this problem by proposing an efficient anomaly detection framework, denoted as DeepGuard, which improves the detection performance of cyberattacks in SDN based networks by adopting a fine-grained traffic flow monitoring mechanism. Specifically, the proposed framework utilizes a deep reinforcement learning technique, i.e., Double Deep Q-Network (DDQN), to learn traffic flow matching strategies maximizing the traffic flow granularity while proactively protecting the SDN data plane from being overloaded. Afterwards, by implementing the learned optimal traffic flow matching control policy, the most beneficial traffic information for anomaly detection is acquired at runtime-thereby improving the cyberattack detection performance. The performance of the proposed framework is validated by extensive experiments, and the results show that DeepGuard yields significant performance improvements compared to existing traffic flow matching mechanisms regarding the level of traffic flow granularity. In the case of distributed denial-of-service (DDoS) attacks, DeepGuard achieves a remarkable attack detection performance while effectively preventing forwarding performance degradation in the SDN data plane.