Improved Monte Carlo Tree Search for Virtual Network Embedding

In this paper, we consider the Virtual Network Embedding (VNE) problem for 5G networks slicing. This consists in optimally allocating multiple Virtual Networks (VN) on a substrate virtualized physical network while maximizing among others, resource utilization, maximum number of placed VNs and network operator's benefit. We solve the online version of the problem where slices arrive over time. We propose the use of the Nested Rollout Policy Adaptation (NRPA) algorithm, a variant of the well known Monte Carlo Tree Search (MCTS). Both algorithms learn by randomly simulating the embedding, but NRPA also learns how to perform better simulations over time. Performance analysis with different scenarios, show that NRPA improves acceptance and reward ratios (by up to 69% and 65%). We also show how a smart initialization of the learning process can help improve the results furthermore (up to a 12.5% increase of acceptance ratio).