Online Fault-tolerant VNF Chain Placement: A Deep Reinforcement Learning Approach

Since Network Function Virtualization (NFV) decouples network functions (NFs) from the underlying dedicated hardware and realizes them in the form of software called Virtual Network Functions (VNFs), they are enabled to run in any resource-sufficient virtual machines (VMs) and offer diverse network services by service function chains (SFCs). Given the complexity and unpredictability of the network state, we propose a deep reinforcement learning (DRL) based online SFC placement method named DDQP (Double Deep Q-networks Placement). Meanwhile, VNFs are vulnerable to various faults such as software failures. Thus, we backup standby instances to enhance the fault tolerance of our model, and DDQP automatically deploys both active and standby instances in real-time. Specifically, we use DNN (Deep Neural Networks) to deal with large continuous network state space. In the case of stateful VNFs, we offer constant generated state updates from active instances to standby instances to guarantee seamless redirection after failures. With the goal of balancing the waste of resources and ensuring service reliability, we introduce five progressive schemes of resource reservations to meet different customer needs. Our experimental results demonstrate that DDQP responds rapidly to arriving requests and reaches near-optimal performance. Specifically, DDQP outweighs the state-of-the-art method by 16.30% higher acceptance ratio with 82x speedup on average.