An Online Data Deduplication Approach For Virtual Machine Clusters

With the popularity of cloud computing paradigm, virtual machine clusters (VMC) emerge and become the main container of distributed applications. Taking distributed snapshot of VMC are widely used to ensure system reliability accordingly. However, due to the heavyweight nature of virtual machine technology, a large amount of space is consumed when taking snapshot of VMC. To address the above issues, we propose an online deduplication mechanism which aims at improving storage efficiency without sacrificing the performance of VMC. "Online" means that duplicated memory pages are detected and merged before being saved into snapshot files. In this way not only the VMC snapshot size but also the I/O bandwidth consumption are reduced. A Save-Locally-Compare-Globally (SLCG) strategy is designed to guarantee an optimal deduplication ratio and minimized network overhead. A fast duplicated page searching algorithm is proposed to speed up the process of finding duplicated pages and reduce the performance overhead. A prototype of SlimVMC has been implemented on KVM and the experimental results show the effectiveness and efficiency of our system.