Autonomic Elasticity Control for Multi-Server Queues Under Generic Workload Surges in Cloud Environments

Cloud computing environments and Internet datacenters consist of a multitude of servers that process user requests. Performance and scalability can suffer greatly when the workload surges to levels that cause a system to become unstable (i.e., when the arrival rate of requests exceeds the system’s capacity to process them). This article presents a detailed design and evaluation of an autonomic elasticity controller for surges of any shape. This controller uses an analytical model, derived by the authors, of a single-queue multiple-server system (G/G/c) subject to workload surges that cause the system to become unstable during finite time intervals. The controller is evaluated through extensive simulations and by using publicly available Google traces. The controller is further extended to take into account VM startup delays. The article also illustrates how fudge factors can be used to more aggressively react to surges at the expense of additional resources. Finally, our controller is compared with a hypothetical oracle controller that knows the exact shape of the surge when it starts to occur.