Unitizing Performance of IaaS Cloud Deployments

In order to establish a broader market for cloud computing, offers must be made comparable. Several efforts exist to compare performance of products from different providers and convey an idea of what to expect by means of (periodical) reports. Yet, buying IaaS cloud compute resources remains a blind bargain. The actual performance of a customer's deployment may, for various reasons, be substantially different from such third-party reports. Particularly, a cloud user cannot rely on receiving the same performance, be it because of higher load or arbitrary cloud reconfiguration. To render service levels of different cloud products meaningful and comparable within and across providers, these will have to commit themselves to providing performance according to some reference measure that also regards virtualization, resource allocation and isolation. Though the actual benchmarks will likely differ across application and market niches, the methodology to define, measure and guarantee performance remains the same. In this paper we propose a method for quantifying, determining and ensuring performance on the basis of a performance unit that conveys what performance can be expected from a VM deployment and is suitable for use in SLAs. The abstract approach is exemplified and validated by a case study with concrete benchmarks on a KVM-based cloud.