Joint Optimization of Microservice Deployment and Routing in Edge via Multi-Objective Deep Reinforcement Learning

Edge computing technologies with container-based microservice architectures promise to provide stable and low-latency services for large-scale and complex edge applications. However, due to the limited CPU and storage resources in edge computing scenarios, the coarse-grained service deployment on edge nodes causes performance bottlenecks. In addition, the effective deployment of microservices is tightly correlated with request routing, but the current research ignores the joint optimization of multi-instance deployment and routing. In this paper, we first model the problem of jointly optimizing service deployment and routing in a dynamically changing environment with multi-edge network collaboration based on a queuing network analysis. Secondly, we design heuristic algorithms to scale microservice instances horizontally in dynamic user request states. In addition, we propose a reinforcement learning algorithm based on reward shaping (RSPPO) to minimize user waiting delay and edge network resource consumption. We also solve the microservice deployment and request routing problem for multi-edge collaboration to achieve load balancing among edge nodes. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.