Multi-Agent DRL-Based Deadline-Aware Routing for Deterministic Wide-Area Networks

The advancement of 6G technologies has sparked interest in the study of low-latency and deterministic communications. A critical challenge for providing an end-to-end deterministic service in 6G is how to overcome the high volume and time-varying properties of wide-area network (WAN) traffic to achieve deterministic support over WANs. In this paper, we propose a deadline-aware framework to support deterministic communication over WANs, where a novel coordinated earliest deadline first scheduler is utilized in the data plane to guarantee timely transmission. Specifically, we formulate the deadline-aware routing problem in the proposed framework as a stochastic optimization problem to maximize the timely delivery ratio. To improve scalability, we propose a deadline-aware routing algorithm based on multi-agent deep reinforcement learning that computes a deadline-satisfied route in a hop-by-hop manner. Simulation results validate that the proposed algorithm can achieve a higher timely delivery ratio, lower latency, and lower jitter compared with benchmark routing algorithms and mainstream deterministic schemes.