Efficient Cooperative Mechanism for Distributed Multi-Agent Traffic Signal Control

Traffic signal control (TSC) provides a cost-effective approach to alleviating urban traffic congestion without requiring modifications to physical road infrastructure. As a classic multi-agent coordination problem, TSC can be optimized with multi-agent reinforcement learning (MARL). However, understanding the cooperative strategies among traffic signals is challenging due to the spatiotemporal dynamics inherent in distributed traffic signal controllers. Previous MARL approaches for TSC often rely on imprecise and redundant cooperation mechanisms, leading to excessive communication costs and inefficient training. In this study, we propose a novel dynamic and controllable cooperation MARL algorithm (DCoo) to optimize traffic signals coordination for alleviating traffic congestion. DCoo introduces a collaboration scheduling module (CSM) to obtain spatiotemporal collaborative information and incorporates a cooperative hyper-decision network (HDN) to optimize decisions by leveraging the processed information. In particular, the CSM first captures crucial temporal information for cooperation by analyzing the relationships between historical and current observations of intersections. CSM then frames the selection of spatial information as a binary classification problem to effectively expand the scope of collaboration among intersections. Additionally, HDN integrates an attention mechanism and permutation invariance to enhance the scalability of DCoo. We conduct comparative experiments on three real-world traffic datasets, and the results demonstrate that DCoo outperforms comparable algorithms in both training and testing phases.