Hierarchical Scaling of Microservices in Kubernetes

In the last years, we have seen the increasing adoption of the microservice architectural style where applications satisfy user requests by invoking a set of independently deployable services. Software containers and orchestration tools, such as Kubernetes, have simplified the development and management of microservices. To manage containers' horizontal elasticity, Kubernetes uses a decentralized threshold-based policy that requires to set thresholds on system-oriented metrics (i.e., CPU utilization). This might not be well-suited to scale latency-sensitive applications, which need to express requirements in terms of response time. Moreover, being a fully decentralized solution, it may lead to frequent and uncoordinated application reconfigurations. In this paper, we present me-kube (Multi-level Elastic Kubernetes), a Kubernetes extension that introduces a hierarchical architecture for controlling the elasticity of microservice-based applications. At higher level, a centralized per-application component coordinates the run-time adaptation of subordinated distributed components, which, in turn, locally control the adaptation of each microservice. Then, we propose novel proactive and reactive hierarchical control policies, based on queuing theory. To show that me-kube provides general mechanisms, we also integrate reinforcement learning-based scaling policies. Using me-kube, we perform a large set of experiments, aimed to show the advantages of a hierarchical control over the default Kubernetes autoscaler.