Towards Performance Modeling as a Service by Exploiting Resource Diversity in the Public Cloud

Cloud computing platforms such as Amazon EC2, Google Computing Engine, and Microsoft Azure offer dozens of virtual machine (VM) types with a wide range of resource capacity vs. price trade-offs, requiring a customer to consider numerous resource configurations when evaluating service needs. This report investigates the possibility of using the diversity of VM types to predict the performance of new VM types using black box modeling. The performance model used is a multiple linear regression of the average server response time, server load (throughput in requests per second), the number of CPU cores, and the memory in the procured VM. For three commonly used database servers - Redis (key-value stores), Apache Cassandra (NoSQL) and MySQL - the model accuracy increases for larger sets of VMs. e.g., for Redis, the measure of model efficacy improves from 0.4-0.5 with 2 VM types for training and 0.7 for 3 VM types to 0.8 for 4 VM types. These results suggest further interesting research challenges, such as the possibility of automating the process of calibrating performance models using diverse resource types on a public cloud leading to "performance modeling as a service".