Access Control in Satellite-Terrestrial Backhaul Network: An Emergent Communication Based Multi-Agent Reinforcement Learning Approach

The satellite-terrestrial backhaul network (STBN) has recently emerged as a promising architecture to provide cost-effective seamless communication services to mobile users, especially in remote areas. In such networks, limited backhaul capacity, dynamic channel states, and user mobility pose significant challenges to user access control, involving efficient channel allocation and interference coordination to maximize user quality of service and network resource utilization. This dynamic user access problem can be formulated as a sequential decision process, and thus multi-agent reinforcement learning (MARL) can be a potential solution. However, conventional MARL lacks cooperation mechanisms for accurate state inference, which is critical for making optimal and cooperative resource allocation and interference management actions in such a dynamic and resource-constrained STBN. Moreover, although centralized coordination may improve system performance, it raises privacy and scalability concerns. In this paper, we propose a decentralized cooperative user access strategy, by leveraging emergent communication based MARL (EC-MARL). EC-MARL incorporates an emergent communication mechanism into the MARL framework, enabling user equipments (UEs) to cooperatively infer the network state more accurately while preserving privacy and scalability. Specifically, each agent encodes its local observation into a cooperation message, and shares the message among other agents. Based on these shared messages and their own local observations, UEs cooperatively make access decisions. Simulation results demonstrate that the proposed EC-MARL approach significantly outperforms several benchmarks in various network settings.