Leveraging CAVs to Improve Traffic Efficiency: An MARL-Based Approach

With the capability of intelligent control and communicating with surrounding vehicles and infrastructures, connected and automated vehicles (CAVs) can drive cooperatively and have more positive effects on traffic efficiency. Cooperative and real-time path planning for CAVs stands as a pivotal solution to mitigate traffic congestion and augment travel efficiency. However, most of the existing path planning schemes predominantly concentrate on minimizing the travel times of vehicles, sidelining the broader issue of alleviating traffic congestion in urban settings. Therefore, in this paper, we propose a novel collaborative vehicle path planning scheme, leveraging the intelligent control and the communicating ability of CAVs. The primary objective is to reduce traffic congestion within the overall transportation system and improve traffic efficiency. Specifically, we focus on a general urban scenario with various types of vehicles, including CAVs, connected vehicles (CVs), and traditional human-driven vehicles (TVs), To enhance traffic efficiency in such a scenario, we design a collaborative path planning scheme to discover the efficient paths for both CAVs as well as CVs. In this scheme, we treat each CAV as an agent and formulate the multiple CAVs' path-planning problem as a Markov game. To solve the above Markov game, we design a multi-agent convolutional attention reinforcement learning (MACA) framework to generate paths with minimal travel time for CAVs. More concretely, the proposed MACA framework incorporates a convolutional neural network (CNN) layer to capture spatial correlation behind traffic conditions. Additionally, a graph attention network (GAT) layer is employed to integrate the influence of neighboring agents during the path-planning process. To further reduce traffic congestion, we extend the MACA framework into a collaborative MACA (C-MACA) scheme in vehicular networks, where CAVs are empowered to periodically broadcast their path information to surrounding CVs, providing valuable insights for their path planning. Subsequently, to prevent new congestion caused by the aggregation of CVs, we design a heuristic algorithm for CVs to make informed path decisions. We build up a simulator based on a real-world city road map and conduct extensive experiments. The experimental results demonstrate that the proposed scheme can decrease CVs' travel time by up to 10.9 % and reduce the average queue length around junctions by up to 6.5 % over several state-of-the-art approaches, without sacrificing the travel efficiency of CAVs.