The vMatrix: server switching

Today most Internet services are pre-assigned to servers statically, hence preventing us from doing real-time sharing of a pool of servers across as group of services with dynamic load. Fluidly copying services in and out of servers remains a challenge due to the many dependencies that such services have on software, hardware, and most importantly, people. In this paper we present a novel solution, which builds on top of the classic operating systems concept of a virtual machine monitor (VMM). A VMM allows us to encapsulate the state of the machine in a virtual machine file, which could then be activated on any real machine running the VHM software. This eliminates the software dependencies problem by allowing us to move the whole machine around including the operating system, libraries, and third party modules that the service depends on. It eliminates the hardware dependencies problem by allowing us to mimic the hardware that the service expects regardless of the real hardware of the hosting machine. It also solves the people dependency problem by presenting the developers and system administrators with the same isolation model that they are used too with statically allocated servers. We describe our vMatrix framework in detail and address how to load balance the virtual machine services across the real-machines to maximize utilization efficiency (in terms of machines and people costs) such that total cost of the system is reduced without degrading the service performance and without requiring cost prohibitive code and architectural changes to existing legacy services. Our solution also offers additional side benefits like on-demand replication for absorbing flash crowds (in case of a newsworthy event like a major catastrophe) and faster failure recovery times.