Heterogeneous Multi-Agent Reinforcement Learning based on Adaptive Curiosity for Traffic Signal Control

Intelligent Traffic Signal Control (TSC) aims to optimize urban traffic management. However, the traditional fixed-cycle intersection signal stands as one of the main factors causing traffic congestion. In addition, with the burgeoning complexity of traffic scenarios and escalating travel demands, TSC becomes a daunting challenge. Recently, the exploration of reinforcement learning in the domain of TSC has emerged as a significant research frontier because it can handle complex scenarios and offers scalability. We follow the framework and propose a multi-agent reinforcement learning (MARL) method-ology, in which we incorporate two pivotal enhancements. First of all, to ensure efficacy and stability of training, our method seamlessly combines the Actor-Critic framework with the dou-ble Q-network. Secondly, we introduce an advanced adaptive curiosity mechanism to exploit intrinsic rewards founded on the agents' varied action space properties, and hence offers a personalized exploration guide for each agent. Our empirical validation, implemented on the Simulation of Urban Mobility (SUMO) platform, covers both homogeneous grid environment and heterogeneous city setting. Notably, in comparison with traditional cyclic scheduling methods and existing advanced MARL solutions, our algorithm exhibited better traffic flow efficiency and robustness.