Deep-NFVOrch: leveraging deep reinforcement learning to achieve adaptive vNF service chaining in DCI-EONs

Due to the rise of cloud computing, how to realize adaptive and cost-effective virtual network function service chaining (vNF-SC) in a datacenter interconnection based on an elastic optical network (DCI-EON) has become an interesting but challenging problem. In this work, we tackle this problem by optimizing the design of a deep reinforcement learning (DRL)-based adaptive service framework, namely, Deep-NFVOrch. Specifically, Deep-NFVOrch works in service cycles and tries to reduce the setup latency of vNF-SC by invoking request prediction and pre-deployment at the beginning of each service cycle. We introduce a DRL-based observer (DRL-Observer) to select the duration of each service cycle adaptively according to the network status. The DRL-Observer is designed based on the advantage actor critic, which can interact with the network environment constantly through its deep neural network and learn how to make wise decisions based on the environment’s feedback. Our simulation results demonstrate that the DRL-Observer converges fast in online training with the help of a few asynchronous training threads, and the Deep-NFVOrch with it achieves better performance than several benchmarks, in terms of balancing the tradeoffs among overall resource utilization, vNF-SC request-blocking probability, and the number of network reconfigurations in a DCI-EON.