A Multi-Agent Federated Reinforcement Learning-Based Cooperative Vehicle-Infrastructure Control Approach Framework for Roundabouts

In contrast to autonomous driving (AD) systems based on a single vehicle, the intelligent vehicle infrastructure cooperative system enables intelligent connected vehicles (ICVs) to perform active safety control and road collaboration management through vehicle-Infrastructure dynamic information interaction. However, complex intersections involve a wide range of interactions between agents, traffic participants and road infrastructures. The presence of time-varying traffic conditions and redundant traffic information may impede the efficiency and effectiveness of the vehicle-infrastructure system. Meanwhile, existing cooperative systems typically assume voluntary client access and data sharing, overlooking the information asymmetry challenges stemming from privacy awareness and the model interpretability challenges associated with data-driven approaches. This paper proposes a federated reinforcement learning (FRL) architecture for vehicle-infrastructure cooperative control is predicated on data and rule fusion, with the objective of considering privacy awareness and sample efficiency by employing a screening-based dual aggregation federated learning (FL) method. The proposed architecture enhances the interpretability of the algorithmic framework at four perspectives: modeling, training, optimization, and application. In this paper, we initially undertake training and performance comparisons on the OpenDD real traffic roundabout road driving dataset, and then extend the proposed method to more complex Nocrash benchmarks and real-world campus traffic environment for algorithm scalability validation.