Deep Reinforcement Learning for Dynamic Task Scheduling in Data Relay Satellite Networks

Data Relay Satellite Networks (DRSNs) face the challenge of balancing increasing relay mission demands with limited resources. Dynamic occurrences further complicate this challenge, particularly in addressing large-scale problems. Therefore, this paper proposes an end-to-end deep reinforcement learning (DRL) approach to maximize total task profits within dynamic DRSN environments. A continuous-time Markov decision process model is formulated to effectively represent the problem, significantly reducing the length of the solution sequence without degrading scheduling quality. Additionally, an interactive training framework comprising two phases—pre-training and intensive training—is proposed to enhance the model's performance and generalization capability. In both phases, populations are employed instead of single-shot inference, providing a more efficient approach to dynamic environment exploration. Once trained, the policy network can solve problems of varying sizes in static and dynamic benchmark tests. Experimental results demonstrate that the proposed method, with its strong generalization capability across different problem sizes, outperforms state-of-the-art DRL and heuristic methods regarding total task profits, running time and completion rate.