PipeDevice

Containers have become prevalent in public clouds due to the performance, portability, and deployment benefits compared to virtual machines [1, 8, 14]. They support a wide variety of workloads, from microservices to data analytics and machine learning. Containerized applications often entail extensive bulky data transfers to exchange intermediate results of data processing among peers. Examples include the shuffle stage in MapReduce jobs [3, 15, 16, 23] and the model update process with parameter server and allreduce in distributed machine learning [5, 19, 32, 35].