Toward Reliability-Enhanced, Delay-Guaranteed Dynamic Network Slicing: A Multiagent DQN Approach With an Action Space Reduction Strategy

Network availability and service continuity are major concerns for network operators to provide reliable communication services for Internet of Things (IoT), which are particularly challenging to achieve in virtualized network slicing environment where network services are exposed to the failure risks of both software (virtual network function (VNF) instances) and hardware (physical nodes). In general, the redundancy-based VNF backup solutions are used to improve the reliability of virtualized network slices. However, backup VNFs require the same amount of resources as the primary VNFs, which will result in high-resource cost. In this article, we propose a joint VNF partition and hybrid backup scheme for VNF orchestration, backup and mapping, whose aim is to construct the reliability-enhanced and delay-guaranteed network slices at minimum cost. Specifically, the VNF partition method divides a single VNF into multiple thinner VNFs with lower processing capacity and is expected to enhance the reliability of network slices with less additional resources. The hybrid backup scheme includes both onsite and offsite backup forms. Then, considering the time-varying network environment and IoT service requirements, we formulate the VNF orchestration, backup and mapping as a dynamic mixed integer linear programming (DMILP) problem, and model the dynamic problem as a Markov decision process (MDP). In view of the large action space of the formulated MDP, we propose a multiagent deep reinforcement learning (DRL) approach with an action space reduction strategy to achieve the dynamic VNF orchestration, backup and mapping solution. Simulation results demonstrate that the proposed joint VNF partition and hybrid backup scheme can obtain superior delay and reliability performance with low-network cost.