Distributed Autonomous Virtual Resource Management in Datacenters Using Finite-Markov Decision Process

To provide robust infrastructure as a service, clouds currently perform load balancing by migrating virtual machines (VMs) from heavily loaded physical machines (PMs) to lightly loaded PMs. Previous reactive load balancing algorithms migrate VMs upon the occurrence of load imbalance, while previous proactive load balancing algorithms predict PM overload to conduct VM migration. However, both methods cannot maintain long-term load balance and produce high overhead and delay due to migration VM selection and destination PM selection. To overcome these problems, in this paper, we propose a proactive Markov Decision Process (MDP)-based load balancing algorithm. We handle the challenges of allying MDP in virtual resource management in cloud datacenters, which allows a PM to proactively find an optimal action to transit to a lightly loaded state that will maintain for a longer period of time. We also apply the MDP to determine destination PMs to achieve long-term PM load balance state. Our algorithm reduces the numbers of service level agreement (SLA) violations by long-term load balance maintenance, and also reduces the load balancing overhead (e.g., CPU time and energy) and delay by quickly identifying VMs and destination PMs to migrate. We further propose enhancement methods for higher performance. First, we propose a cloud profit oriented reward system in the MDP model so that when the MDP tries to maximize the rewards for load balance, it concurrently improves the actual profit of the datacenter. Second, we propose a new MDP model, which considers the actions for both migrating a VM out of a PM and migrating a VM into a PM, in order to reduce the overhead and improve the effectiveness of load balancing. Our trace-driven experiments show that our algorithm outperforms both previous reactive and proactive load balancing algorithms in terms of SLA violation, load balancing efficiency, and long-term load balance maintenance. Our experimental results also show the effectiveness of our proposed enhancement methods.