A SmartNIC-based Load Balancing and Auto Scaling Framework for Middlebox Edge Server

Edge Cloud servers running network functions for cellular carriers must provide both high performance and high resource efficiency. Recent research has focused on load balancing across multiple servers in a large data center, yet load balancing within a single host has been neglected. Multi-core servers make use of multi-queue NICs in order to distribute incoming packets to different CPUs; however, existing approaches can lead to an unfair distribution of work since different flows may have significant variations in size and processing time. While the NIC has the detailed information about incoming traffic, it lacks knowledge about the server’s resources which prevents the hardware from fully supporting auto scaling and load balancing features. These problems are exacerbated for edge middlebox servers running virtual network functions, where processing latency is critical.SmartNICs offer a unique opportunity to resolve this problem with their increased programmability and flexibility. In this paper we present SmartLB - a programmable hardware framework to explore how middlebox servers can work together with SmartNICs to provide a cross-layer solution. Our evaluation shows that SmartLB can increase flow-level fairness, reduces tail latency, and support auto scaling feature for high performance network functions.