Maxwell’s Demon in Tail-tolerant, Resource-efficient Serverless Computing

Computing systems always face a “resource allocation dilemma” that shows the great difficulties in trading off resource efficiency for tail latency, due to the internal uncertainty of cluster status and execution behavior. Inspired by the imaginary “Maxwell’s demon” in thermodynamics who can reduce the uncertainty through a per-gas molecule-level control policy, we consider the “one-to-one mapping” feature of serverless computing and build a novel resource allocator, named Maxwell, that can achieve low tail latency and high resource efficiency in serverless simultaneously. Like the “Maxwell’s demon Maxwell is able to optimize the resource allocation for every request. It observes the state of each request and makes decisions about the minimum resource allocation through a reinforcement learning predictor. As the per-request-grained control incurs significant overhead, we further design a pipeline for avoiding the accumulated effect on a workflow. Experimental results show that Maxwell not only saves up to 31% CPU resources but also reduces the standard deviation of latency by 1.9×. Its time overhead is negligible and the resource overhead is also limited when the query per second $\leq$500.