Deep Reinforcement Learning based Dynamic Edge/Fog Network Slicing

To accommodate increasingly diverse traffic demands in future 6G wireless networks, intelligent and dynamic network slicing schemes are needed to exploit available edge/fog resources (i.e., radio, computing, storage resources). In this paper, we present a new dynamic edge/fog network slicing scheme (EFNS) in which tenants can temporarily lease back to the Infrastructure Provider (InP) unused resources to serve demands exceeding its current resources in stock. To efficiently use the resources, tenants can also lease their subscribers' terminals when idle to act as fog nodes augmenting the infrastructure and service capability of the InP. Our goal is to find an optimal slice request admission policy, which includes slices with augmented resources, to maximize the long-term revenue of the InP. A semi-Markov decision process is used to model the arrival of slice requests by taking into account the dynamics of users' demands and availability of resources. To find the optimal policy under uncertain resource demands, a Q-learning (Q-EFNS) algorithm is developed. Additionally, to improve the convergence time and reduce the computational complexity of Q-learning in large-scale scenarios, a Deep reinforcement learning (DQ-EFNS) algorithm and an enhancement based on a Deep Dueling (Dueling DQ-EFNS) algorithm are presented. Finally, simulation results show improvements in the range between 20% to 60% by our algorithms compared to conventional fixed network slicing when only 8% of the subscribers act as fog nodes. Besides, by using Dueling DQ-EFNS, the optimal slice request admission policy is obtained in just a few iterations.