Using Deep Reinforcement Learning for Routing in IP Networks

This paper proposes Trailnet, a deep reinforcement learning approach to predict the output port (of a router) for an IP packet based on its destination IP address. Trailnet attempts to replace the forwarding table at an IP router with a computational model. To optimally learn each router's forwarding decisions, we propose to train the Artificial Neural Network (ANN) of Trailnet with value iteration and stochastic gradient descent. Through the value iteration algorithm, Trailnet estimates the cost of IP packet forwarding along different ports of a router and eventually selects a port that optimizes a cost function. We evaluate the generalization capability of the ANN on two sufficiently large service provider's network topologies containing millions of IP addresses. Our evaluations show that Trailnet achieves high accuracy and fast inference time for predicting the correct output ports for incoming IP packets. Our results support the claim of replacing forwarding tables and distributed protocols (for computing shortest paths) with a computation model while operating at a high line rate in IP routers.