Congested Urban Networks Tend to Be Insensitive to Signal Settings: Implications for Learning-Based Control

This paper highlights several properties of large urban networks that can have an impact on machine learning methods applied to traffic signal control. In particular, we note that the average network flow tends to be independent of the signal control policy as density increases past the critical density. We show that this property, which so far has remained under the radar, implies that no control (i.e. a random policy) can be an effective control strategy for a surprisingly large family of networks, especially for networks with short blocks. We also show that this property makes deep reinforcement learning (DRL) methods ineffective when trained under congested conditions, independently of the particular algorithm used. Accordingly, in contrast to the conventional wisdom around learning-based methods promoting the exploration of all states, we find that for urban networks it is advisable to discard any congested data when training, and that doing so will improve performance under all traffic conditions. Our results apply to all possible grid networks thanks to a parametrization introduced here. The impact of the turning probability was found to be very significant, in particular to explain the loss of symmetry observed in the macroscopic fundamental diagram of the networks, which is not captured by existing theories that rely on corridor approximations without turns. Our findings also suggest that supervised learning methods have enormous potential as they require very little examples to produce excellent policies.