Fair Virtual Network Function Mapping and Scheduling Using Proximal Policy Optimization

Network function virtualization redefines a network service as a softwarized chain of virtual network functions (VNFs), which decouples the specific network service from dedicated devices and greatly reduces the hardware cost. The VNFs are generally deployed on common off-the-shelf servers and statistically share computational resources. Due to the inherent integer constraints, the corresponding VNF mapping and scheduling issue is a highly challenging task. In this paper, we consider the mapping and scheduling of VNFs for a given network service, for which a flexible job shop scheduling problem is formulated to optimize the max-min fairness while ensuring the delay requirements of different service chains. Specifically, we propose a deep reinforcement learning method based on offline proximal policy optimization, which dynamically determines the mapping and scheduling decision based on the state of unfinished service chains. The proposed algorithm is scalable to the number of service chains and can be enhanced by Monte Carlo tree search. Numerical results show that the proposed algorithm outperforms the traditional random forest and the greedy algorithms in terms of both service fairness and acceptance ratio.