Joint Optimization of Spectrum and Power for Vehicular Networks: A MAPPO based Deep Reinforcement Learning Approach

This paper focuses on the issue of resource allocation in vehicular networking, specifically in scenarios where vehicle-to-vehicle (V2V) links coexist with vehicle-to-infrastructure (V2I) links that utilize pre-allocated spectrum resources for communication. Different link has different quality-of-service (QoS). The V2I links commonly provide high data rate service, while the V2V links usually offer reliable safety message exchange service. However, due to the high mobility of vehicles, the communication network topology is constantly changing, which imposes a significant challenge on ensuring QoS for both V2I and V2V links with minimal overhead. To address this, we propose a joint spectrum and power allocation scheme based on the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm, which has demonstrated excellent performance in multi-intelligent cooperation scenarios and can facilitate the learning process for agents without requiring global information. Furthermore, the D-MAPPO and C-MAPPO based allocation schemes are presented for two different cases, discrete finite power choices and continuous power, respectively. To deal with the non-stationarity during training for multi-agent cooperation, the method of general advantage estimation is utilized to enhance the robustness of proposed algorithms. Finally, simulation results are given to indicate that our schemes can effectively ensure a high capacity of V2I links while guaranteeing stringent low latency and high reliability for V2V links.