Deep Reinforcement Learning for Demand-Aware Joint VNF Placement-and-Routing

One main challenge facing the deployment of the network function virtualization is the resource allocation on demand. As the primary stage of the resource allocation, the effective virtual network function (VNF) placement-and-routing (P&R) is particularly difficult. To this end, we propose a deep reinforcement learning (DRL)-based approach for solving the joint VNF P&R problem subject to diverse service demands. Our approach exhibits strong stability and excellent load balancing ability. It can also effectively counter the non-uniformity of P&R policies. We first introduce the service request model, and formulate the VNF P&R problem, while taking both the resource consumption cost and service delay into the optimization objective. A demand-aware factor is also employed to characterize the diversity of service demands. In the considered problem, the DRL-based P&R scheme is presented, invoking the deep deterministic policy gradient as the main learning algorithm. According to the results of extensive experiments made on the realistic "COST266" network topology, our scheme can approach the optimum, and is superior to existing heuristic algorithms.