Task Scheduling Under a Novel Framework for Data Relay Satellite Network via Deep Reinforcement Learning

Data relay satellite networks (DRSNs) face the challenge of increasing relay mission demands in space networks. To improve the task scheduling efficiency of DRSN further, we propose a novel task scheduling framework, wherein a scheduling sequence is generated by selecting one antenna and selecting one task for the antenna in each step. Subsequently, the task scheduling problem of DRSN (TSPD) is regarded as a sequential decision-making problem and is optimized using a method based on deep reinforcement learning (DRL), which overcomes the difficulty of designing heuristics with massive efforts. In this study, a mathematical model and corresponding Markov decision model based on our proposed scheduling framework are constructed, and for the first time, a policy network that includes one encoder based on the attention mechanism and two decoders is designed to solve the TSPD. In addtion, extensive experiments are conducted to verify the effectiveness of our scheduling framework and demonstrate that the DRL method can obtain a scheduling scheme with the highest profits with decent generalization to different task scales, number of user spacecrafts and execution duration of tasks.

I. Introduction

In Recent years, data relay satellite networks (DRSNs) have been applied in many fields, such as manned spaceflight missions [1], tracking, telemetry and command (TT&C) demands of aerospace vehicle missions [2] and data relay missions for Earth observation satellites. A DRSN is mainly composed of tracking and data relay satellites (TDRSs) synchronous with the geostationary orbit, user spacecrafts (USs) in the middle and low orbits, and ground terminals (GTs), in which TDRS as a space-based transmission platform plays an important role in data transmission, continuous tracking and orbit control for the US [3], [4], [5]. However, TDRS cannot communicate with the US all the time because of the masking of the Earth, and only when the inter-satellite link antenna beam (ILAB) between them is available, the TDRS can execute the task of the US. In addition, as the number and types of spacecrafts in space increase, DRSNs undertake more missions from a variety of users [6], [7], [8]. Therefore, extensive user demands cannot generally be fully satisfied by the limited ILAB resources in a DRSN [9]. Consequently, it is of great significance to allocate limited ILAB resources to the tasks of the USs appropriately to obtain an effective task scheduling scheme of DRSN, which has attracted increasing attention in recent years [10], [11], [12], [13], [14].