Service Function Chain Deployment Using Deep Q Learning and Tidal Mechanism

With the rapid development of software-defined networking/network function virtualization (NFV) technologies, service function chaining (SFC) has become a key enabler for end-to-end service provisioning in future networks. In the Internet of Things (IoT), the highly dynamic nature of the network environment demands flexible and adaptive mechanisms for dynamic SFC deployment to fully utilize network resources while meeting the service requirements. Although reinforcement learning (RL) techniques offer a promising approach to dynamic SFC deployment, the learning delay of RL may limit its prompt response to sudden changes in network state and/or service demand. To address this challenge in this article, we propose to employ a deep
$Q$
-learning network (DQN) method for dynamic SFC deployment combined with a tidal virtual machine (TVM) control mechanism for adaptive virtual machine (VM) auto-scaling. We present a tidal DQN framework (TDQNF) that integrates the DQN method and TVM control in the ETSI NFV architecture and develop the algorithms for implementing DQN-based decisions for SFC deployment and TVM control for VM scaling. The performance of the TDQNF framework with the proposed algorithms has been evaluated through extensive simulation experiments. The obtained experimental results verify the effectiveness of the proposed scheme and indicate better performance in terms of system delay, packet loss, and load balancing in large-scale networks compared to existing methods.