A Reinforcement Learning Approach for Placement of Stateful Virtualized Network Functions

Network softwarization increases network flexibility by supporting the implementation of network functions such as firewalls as software modules. However, this creates new concerns on service reliability due to failures at both software and hardware level. The survivability of critical applications is commonly assured by deploying stand-by Virtual Network Functions (VNFs) to which the service is migrated upon failure of the primary VNFs. However, it is challenging to identify the optimal Data Centers (DCs) for hosting the active and stand-by VNF instances, not only to minimize their placement cost, but also the cost of a continuous state transfer between active and stand-by instances, since a number of VNFs are stateful. This paper proposes a reinforcement learning (RL) approach for the placement of stateful VNFs that considers a joint reservation of primary and backup resources with the objective of minimizing the overall placement cost. Simulation results show that the proposed algorithm is optimized in terms of both acceptance ratio and cost, resulting in up to 27% and 30% improvements in terms of accepted requests and placement cost compared to a state-of-the art algorithm.