DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Multi-beam satellite communication systems have been widely recognized as an efficient technology for providing reliable and high-speed communication services. In this paper, we consider a multi-beam satellite communication system, which consists of a multi-beam satellite, ground cells and a ground gateway for processing information of the system. We focus on the beam scheduling, subchannel and power allocation problem to improve system performance. To jointly consider data transmission performance and power consumption, we define a utility function as the weighted sum of service queue length and satellite transmit power. To adapt to the dynamic arriving of data packets and the time-varying satellite channels, we formulate the resource allocation problem as a long-term utility function maximization problem. Since the optimization problem is a non-convex mixed integer problem, which cannot be solved using traditional convex optimization tools, we first decouple the original problem into beam scheduling subproblem and joint subchannel and power allocation subproblem. To solve beam scheduling subproblem, two beam scheduling schemes are proposed. Furthermore, three deep reinforcement learning (DRL)-based joint subchannel and power allocation algorithms are proposed to tackle joint subchannel and power allocation subproblem. Numerical results demonstrate the effectiveness of the proposed algorithms.