Memory overbooking and dynamic control of Xen virtual machines in consolidated environments

The newly emergent cloud computing environments host hundreds to thousands of services on a shared resource pool. The sharing is enhanced by virtualization technologies allowing multiple services to run in different virtual machines (VMs) on a single physical node. Resource over-booking allows more services with time-varying demands to be consolidated reducing operational costs. In the past, researchers have studied dynamic control mechanisms for allocating CPU to virtual machines, when CPU is over-booked with respect to the sum of the peak demands from all the VMs. However, runtime re-allocation of memory among multiple VMs has not been widely studied, except on VMware platforms. In this paper, we present a case study where feedback control is used for dynamic memory allocation to Xen virtual machines in a consolidated environment. We illustrate how memory behaves differently from CPU in terms of its relationship to application-level performance, such as response times. We have built a prototype of a joint resource control system for allocating both CPU and memory resources to co-located VMs in real time. Experimental results show that our solution allows all the hosted applications to achieve the desired performance in spite of their time-varying CPU and memory demands, whereas a solution without memory control incurs significant service level violations.