Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals

Information dissemination in mobile networks turns out to be a problem when the network is sparse. Mobile networks begin to establish a separate cluster attributable to the limited communication range of terminals. The multi-beam satellite communication systems can play a significant role in providing direct-to-user satellite mobile services and connecting the separated clusters. This paper focuses on how to efficiently schedule limited satellite-based radio resources to enhance transmission efficiency and meet the requested traffic with low complexity. Taking the inter-beam interference and resource utilization variance into consideration, we build a game-theoretic based model for bandwidth allocation in the forward link. As the size of satellite beams increases, the size of the action space for deep reinforcement learning based on a single agent becomes large, resulting in high time complexity. Thus, we extend the single-agent deep reinforcement learning to the multi-agent context and then propose a cooperative multi-agent deep reinforcement learning method to achieve the optimal bandwidth allocation strategy. Each beam works as a player who is willing to satisfy the request traffic with flexible payloads. We built a multi-beam satellite platform using real historical data. The experimental results show that this approach is capable of enhancing transmission efficiency and can be flexible to achieve the desired goal with low complexity.