Meta-Reinforcement Learning for Centralized Multiple Intersections Traffic Signal Control

Despite the success of reinforcement learning to traffic signal control, conventional reinforcement learning-based methods have mostly tackled the traffic signal control for a single network, which requires a large computational cost, and has insufficient generalization ability to the new environment. To solve the above problems, we propose a gradient-based meta-reinforcement learning method for the centralized traffic signal control to extract meta-knowledge from multiple meta-training tasks and utilize the accumulated meta-knowledge to a new task, thus improving the training efficiency. In addition, to mitigate the curse of dimensionality problem of the centralized control, we design a special agent, which decomposes the search space by using the Divide and Conquer paradigm. To the best of our knowledge, we are the first to combine the centralized control method with the meta-reinforcement learning method. Experimental results show that our method successfully generalizes to multiple unseen traffic networks. Compared with the traditional methods and the state-of-the-art reinforcement learning-based methods, our method achieves higher traffic signal control efficiency, faster convergence speed, and more stable performance.