Adaptive VNF Placement Considering Overall Latency and 5G Wireless Channel Reliability in Industry 4.0: A Reinforcement Learning Based Approach

Industry 4.0 incorporates the integration of cloud computing, Industrial Internet of Things (IIoT), and modern communication technologies within the industrial automation systems. Various devices with different network requirements of high reliability and low latency, rely on connectivity. The 5G and Beyond (B5G) software-defined architecture facilitates Network Function Virtualization (NFV), which is an essential solution for fulfilling these stringent demands. NFV allows for the implementation and control of Virtual Network Functions (VNFs) in dynamic network environments. VNF placement optimization has been extensively studied in the 5G perspective outside the industry environment with a focus on minimizing delay and cost, increasing VNF reliability, and increasing resource efficiency. However, the complex dynamics of the wireless channel in industrial environments have a considerable impact on the essential delay factors that are important for optimizing the deployment of VNFs. This study focuses on modeling a Wireless Sensor Network (WSN) based Industry 4.0 factory automation scenario at mmWave band, formulating an optimization problem to minimize overall delay while considering packet loss rate in the 5G industrial wireless channel. The optimization problem is formulated as a Markov Decision Process (MDP) and two Reinforcement Learning (RL) based algorithms AVP-Q and AVP-DQN are proposed for optimizing the VNF placement. The proposed algorithms are extensively evaluated against the Value Iteration algorithm which assumes a completely known MDP model and two other algorithms from the literature. The simulated results show that AVP-DQN outperforms existing algorithms for this scenario by 39% and 22.6% and the achieved performance is only close to that of the Value Iteration algorithm.