Decentralized Request Dispatch for Edge-Clouds: A Diffusion-Based Reinforcement Learning Paradigm

Edge-cloud systems have the potential to achieve ubiquitous computing by providing services in close proximity to users that submit service requests. The key challenge is how to efficiently orchestrate services and dispatch requests to satisfy the Quality of Service (QoS) requirements of users in dynamic edge-cloud environments. With the benefit of efficiently adapting to uncertainty, Reinforcement Learning (RL) based approaches are proposed to solve the request dispatch problem in edge-cloud environments. However, existing RL based approaches are often constrained by inexpressive policies that make highly suboptimal decisions in the field of request dispatch for edge-clouds. To enhance the effectiveness of RL in guaranteeing the QoS requirements of users, this paper presents D2Sched, a novel scheduling framework that represents the policy networks of Multi-Agent Deep Reinforcement Learning (MADRL) as diffusion models to generate request dispatch decisions. To improve the valid probability of generated dispatch decisions, D2Sched coordinates all agents for the resource competition among different requests by carefully considering the availability of system resources and latency targets of requests. Extensive experiments using synthetic and real traces demonstrate that D2Sched can improve the average system throughput under QoS requirements of users by up to 20.1% compared to representative baselines.