Selective commitment and selective margin: Techniques to minimize cost in an IaaS cloud

Cloud computing holds the exciting potential of elastically scaling computation to match time-varying demand, thus eliminating the need to provision for peak demand. However, the uncertainty of variable loads necessitate the use of margins - servers that must be held active to absorb unpredictable potential load bursts - which can be a significant fraction of overall cost. Further, naively switching to an on-demand cloud model can actually degrade true costs (server costs that would be incurred even if margin costs disappeared) because of the fundamental economic rule wherein on-demand services/goods cost more compared to reserved services/goods where the user bears some commitment. On-demand customers pay a premium in exchange for not undertaking the fixed-cost risk that committed customers undertake. This paper addresses the twin challenges of minimizing margin costs and true costs in an Infrastructure-as-a-Service (IaaS) cloud. Our paper makes the following two contributions. First, rather than use a fixed margin, we observe that the margin may be selectively used depending on load levels. Based on the above observation, we develop ShrinkWrap-opt which is a dynamic programming algorithm that achieves optimal margin cost while satisfying the desired (statistical) response time guarantees. Second, we propose commitment straddling - the selective use of some reserved machines in conjunction with on-demand machines - to achieve optimal true-cost. Simulations with real Web server load traces using the Amazon EC2 cost model reveal that our techniques save between 13% and 29% (21% on average) in cost while satisfying response-time targets.