Latency-Based Benchmarking of Cloud Service Providers

With the ever-increasing trend of migration of applications to the Cloud environment, there is a growing need to thoroughly evaluate quality of the Cloud service itself, before deciding upon a hosting provider. Benchmarking the Cloud services is difficult though, due to the complex nature of the Cloud Computing setup and the diversity of locations, of applications and of their specific service requirements. However, such comparison may be crucial for decision making and for troubleshooting of services offered by the intermediate businesses - the so-called Cloud tenants. Existing cross–sectional studies and benchmarking methodologies provide only a shallow comparison of Cloud services, whereas state-of-the-art tooling for specific comparisons of application-performance parameters, such as for example latency, is insufficient. In this work, we propose a novel methodology for benchmarking of Cloud-service providers, which is based on latency measurements collected via active probing, and can be tailored to specific application needs. Furthermore, we demonstrate its applicability on a practical longitudinal study of real measurements of two major Cloud-service providers – Amazon and Microsoft.