RL with Balanced Reward and Masking Mechanism for Multi-NUMA Virtual Machine Scheduling

With the rapid development of data center and cloud computing, the importance of resource management is increasing in recent years. In this paper, we focus on the virtual machine scheduling problem in resource management, that is, how to allo-cate the limited virtual machine resources when a request arrives. Existing works usually model the virtual machine scheduling problem as a bin-packing problem, but they do not consider the multi-NUMA architecture servers that have been widely used in recent years. Compared with the traditional symmetric multiprocessing architecture, the multi-NUMA architecture has the advantages of faster access time and higher energy efficiency, which is more suitable for dealing with today's scenarios with a large number of computing requirements. In this paper, we first formalize the multi-NUMA virtual machine scheduling problem as a combinatorial optimization problem and explain why it is suitable to be solved by reinforcement learning. A reinforcement learning model called BM-DQN is then proposed, which designs a balanced reward function suitable for the multi-NUMA architecture and adopts a masking mechanism to screen out the optional servers in each step of reinforcement learning, so as to improve the solution performance. Experimental results on the open-source real-world virtual machine scheduling dataset of Huawei Cloud show that BM-DQN is superior to the existing methods in terms of the number of completed requests.