Distributed Cooperative Reinforcement Learning-Based Traffic Signal Control That Integrates V2X Networks’ Dynamic Clustering

With the acceleration of urbanization in the world, urban traffic congestion has become an urgent challenge in most cities. Adaptive traffic signal control is the most approved control method to solve the problem, and accurate real-time traffic information is critical to this solution. This paper presents distributed cooperative reinforcement learning-based traffic control that integrates V2X networks’ dynamic clustering algorithm. To obtain traffic flow information accurately and instantaneously, it is important to improve the cluster stability in V2X networks. A dynamic clustering algorithm is proposed based on the enhanced affinity propagation. The proposed clustering algorithm introduces the initial cluster partition to maintain a proper cluster size and adds the lane and destination factors to improve the cluster's stability. The algorithm can provide efficient and accurate traffic state information to traffic signal controls. By integrating the clustering algorithm, a cooperative reinforcement learning control scheme is proposed to balance the traffic load. To address the tough dimensionality curse of reinforcement learning, a distributed mechanism for intersection cooperation is introduced, and a fast gradient-descent function approximation method is proposed to improve the controls’ real-time performance. The proposed intelligent traffic control scheme that integrates the stable clustering algorithm can effectively improve the traffic throughput, reduce the average waiting time, and avoid congestion. Numerical simulations on real scenarios validate the performance of the proposed approach.