Efficient Joint Beam Pattern and Power Allocation in Beam Hopping LEO Satellite Systems: A Multi-Agent Actor-Critic Approach

Multi-beam low Earth orbit (MB-LEO) satellites are crucial for satellite Internet access, as they simultaneously transmit multiple spot beams to serve numerous ground users. The creation of dispersed beam patterns to avoid interference and the efficient allocation of limited onboard resources significantly impact the quality of communication services. Traditional pre-defined periodic beam hopping (BH) methods struggle with unevenly distributed and time-varying traffic requests. Existing BH resource allocation schemes using reinforcement learning face challenges in avoiding inter-beam interference through location selection and are constrained by action space, complicating model training as user and beam numbers increase. This paper presents a comprehensive framework and its mathematical formulation for efficient BH resource allocation in MB-LEO satellite systems using a multi-agent actor-critic method. To address the multi-objective, multi-dimensional optimization problem, the resource allocation process is divided into two stages. First, the optimal beam pattern is determined to maximize channel capacity and system throughput. Then, a power allocation (PA) agent dynamically distributes power to each beam based on traffic demand, optimizing resource utilization. During the training phase, a dynamic adjustment mechanism guides BH agents to balance throughput maximization and packet loss minimization by adjusting the penalty for dropped traffic requests. Experimental results demonstrate the effectiveness of our method in avoiding inter-beam interference, improving system throughput, reducing latency, and allocating resources based on demand.