Hierarchical Regional Control for Traffic Grid Signal Optimization

Centralized traffic control in a large-scale grid is quite challenging due to the large search space of the policy. To deal with this problem, we propose a hierarchical regional control framework that can learn more quickly and efficiently, based on prior knowledge. Specifically, the traffic at intersections is controlled by local controller based on well-adjusted policies. The coordination of the local controllers is decided by a master controller that is trained by using reinforcement learning. The control of the whole grid is handled solely by learning a master policy. The master controller continuously observes the state of the traffic network and predicts the best possible traffic control strategy for the current state. In this way, the dimension of the action space is dramatically decreased, and it is much easier to explore the optimal policy. We verify our method by implementing a series of experiments in SUMO. The numerical experiments demonstrate that our method outperforms the traditional methods and the new control methods based on deep reinforcement learning in various typical scenarios. We also demonstrate that our method is easy to train and operates robustly.