Deep Reinforcement Learning for Joint Service Placement and Request Scheduling in Mobile Edge Computing Networks

Mobile edge computing aims to provide cloud-like services on edge servers located near Mobile Devices (MDs) with higher Quality of Service (QoS). However, the mobility of MDs makes it difficult to find a global optimal solution for the coupled service placement and request scheduling problem. To address these issues, we consider a three-tier MEC network with vertical and horizontal cooperation. Then we formulate the joint service placement and request scheduling problem in a mobile scenario with heterogeneous services and resource limits, and convert it into two Markov decision processes to decouple decisions across successive time slots. We propose a Cyclic Deep Q-network-based Service placement and Request scheduling (CDSR) framework to find a long-term optimal solution despite future information unavailability. Specifically, to solve the issue of enormous action space, we decompose the system agent and train them cyclically. Evaluation results demonstrates the effectiveness of our proposed CDSR on user-perceived QoS.