Efficient Resource Allocation for Multi-Beam Satellite-Terrestrial Vehicular Networks: A Multi-Agent Actor-Critic Method With Attention Mechanism

With the rapid development of intelligent transportation systems, there is an increasing demand for a variety of vehicular services, such as automated driving assistance, emergency alert, infotainment, etc. However, in some situations (e.g., remote areas or maritime scenarios), the terrestrial networks alone cannot serve the vehicular applications very well due to the infrastructure deployment and maintenance issues. Satellite networks have become an effective supplement to terrestrial networks, which complement well in terms of coverage, flexibility, reliability, and availability. In this paper, we consider the low orbit multi-beam satellite-terrestrial networks to serve for vehicles. We model this problem as a cooperative multi-agent reinforcement learning process, where each beam acts as an agent, and the global bandwidth is cooperatively shared among all the agents. A multi-agent actor-critic method with attention mechanism is proposed to allocate resources for vehicles with strict delay requirements and minimum bandwidth consumption. When allocating bandwidth, the channel efficiency, the angle of the beams and the priorities of requests in different regions are also considered. Centralized training and distributed execution is performed in the training of the agents. Extensive simulation results verify the effectiveness of our proposed method, where all the agents can well cooperative to achieve efficient resource allocation on-demand for the vehicles under strictly limited bandwidth resources.