Queue-Waiting-Time Based Load Balancing Algorithm for Fine-Grain Microservices.

For fine-grain microservices, queue-waiting-time is defined as load index of a server for the first time. However, as internet traffic is bursty, queue-waiting-time is not merely calculated by adding up the normal service time of queued requests due to resource contention. Moreover, normal service time changes over time especially for database-driven web applications. Therefore, an adaptive load balancing algorithm is required. This paper focuses on load balancing algorithms under differentiated requests and heterogeneous servers. In order to solve the tuning problem in load balancing, an online learning algorithm of time weight (OLTW) is designed, which can learn the time-weight of request adaptively. Based on OLTW, a shortest queue-waiting-time load balancing algorithm (SQLB) is then proposed, in this algorithm, an incoming request is dispatched to the server with shortest queue-waiting-time. The experimental results show that 80% prediction values of OLTW have relative error of less than 25%, and SQLB outperforms the classical load balancing algorithms in terms of throughput, mean response time and deadline drop rate.