EdgePro: Adaptive Edge Service Provision via Safe Deep Reinforcement Learning

The edge computing paradigm provides fine-grained and distributed resources to users with low service latency. To further utilize the advantage of edge computing to improve users' satisfaction, it is essential to jointly optimize service deployment, task offloading, and resource allocation. However, this is challenging because of limited edge resources, diverse task demands, and coupled decisions. In this paper, we propose EdgePro, a novel adaptive edge service provision approach based on safe deep reinforcement learning, aiming to maximize user satisfaction while fulfilling multiple constraints including deployment budget and edge server resources. Specifically, we formulate the optimization problem as a constrained Markov decision process. By designing a constraint-penalty function, we transform the original multi-constraint problem into an equivalent single-constraint problem, addressing the training oscillations caused by conflicts in satisfying multiple constraints. To handle the discrete-continuous coupled decisions, we employ multiple deep neural networks for coordinated control. Then, we propose a safe deep reinforcement learning algorithm based on augmented proximal policy optimization, which adaptively solves the formulated problem while satisfying safety constraints. Experimental results show that EdgePro significantly outperforms benchmark approaches in user satisfaction, convergence speed, and satisfying constraints.