Joint Resource Allocation for V2X Communications With Multi-Type Mean-Field Reinforcement Learning

Vehicle-and-infrastructure cooperation is emerging in vehicle-to-everything (V2X) communication to increase traffic efficiency and road safety with advanced services. Static infrastructures like roadside units (RSUs) have the potential to provide stable and high-quality communication services, but suffer overload problems caused by uneven spatiotemporal distribution of vehicles. Unmanned aerial vehicles (UAVs) with high flexibility can establish the line-of-sight (LoS) links but require extra scheduling overheads. Furthermore, the scarce spectrum resources, complex interference, limited energy budgets, and the mobility of automobiles also pose significant challenges. In this paper, we combine mean-field game (MFG) theory with multi-agent reinforcement learning (MARL) to allocate resources for heterogeneous infrastructures in non-orthogonal multiple access (NOMA) V2X communication networks. First, a joint sub-band scheduling, transmit power allocation, and UAV deployment problem is addressed, aiming to jointly optimize communication resources for heterogeneous infrastructures under power and QoS constraints. Subsequently, considering the differences among nodes, multi-type agents are designed and applied MARL to get self-learning ability and collaboration. Moreover, MFG theory is employed to tackle the tremendous overhead caused by agent interactions in MARL. Finally, simulation results demonstrate that our proposed method outperforms two state-of-the-art resource allocation schemes in both average energy efficiency and probability of failure.