Accelerating Distributed Cloud Storage Systems with In-Network Computing

A distributed cloud, connecting multiple smaller and geographically distributed data centers, can provide a significant alternative to the traditional model of massive and centralized data centers. Erasure coding is a key solution for improving the efficiency of storage resources in a distributed cloud. However, current end-side based erasure coding systems require significant computing resources because they involve complex calculations. In addition, reconstructing missing data blocks using erasure coding consumes significantly more network bandwidth than replication. Previous studies have focused on designing new transmission and coding schemes to balance the tradeoff between data reliability and various overheads. This study introduces INC-EC, a programmable data plane based in-network erasure coding system. It aggregates multiple data streams within the network, reducing the CPU consumption of host codecs and the bandwidth consumption of the network. The results of the evaluation indicate that it is feasible to deploy INC-EC in hardware programmable switches and that it effectively increases the erasure coding throughput while eliminating redundant cross-rack traffic.