Learn to Optimize: Adaptive VNF Provisioning in Mobile Edge Clouds

Machine learning (ML) has been penetrating into our daily life by facilitating many daily applications, e.g., self-driving, cloud gaming, product fault detection and drones. Meanwhile, there is an emerging trend that adopts ML methods into network optimization problems, such as flow classification, traffic engineering, routing, and etc. Conventional ML methods need careful training for a specific application of a given network structure, and the trained model normally cannot be applied to other applications and network structures. In this paper, we aim to design adaptive ML methods for network optimization problems, with the trained models having the ability of being deployed to any similar problems. In particular, we consider the virtualized network function (VNF) provisioning problem as our target optimization problem. We first propose a deep Q-learning-based optimization framework for VNF provisioning in a mobile edge network with network capacity constraints, by devising an adaptive graph feature embedding method. We then propose a series of deep Q-learning based learning algorithms for the problems of service chaining and the throughput maximization, based on the proposed learning-based optimization framework. We also propose a novel design of master-slave dual neural network that enables the decisions on both cloudlet selections and routing path finding. To stabilize and accelerate the convergence of the proposed methods, we devise a novel environment generation and termination strategy and a new structure for the replay buffer. We also evaluate the performance of the proposed framework and algorithms by extensive simulations. Results show that the proposed algorithms outperform existing methods by around 12%, and the trained model in a network can be directly adapted to other network structures and settings.