Traffic Signal Control for Large-Scale Road Networks Based on Deep Reinforcement with PSR

This paper investigates the problem of Traffic Signal Control (TSC) in large-scale road networks. In extensive road networks, it is customary to define each intersection as an agent, however, the issue of partial observability is particularly prominent. In this paper, Predictive State Representation (PSR) is employed to address the challenge of partial observability in large-scale multi-agent systems. A Multi-agent Deep Reinforcement Learning (DRL) model based on PSR called PSR-XLight is proposed in Large-Scale TSC Systems. Multi-agent PSR is conducted with centralized training and independent filtering which overcome the challenge of prohibitive computations when the number of agents is large. Parameters sharing is adopted between each agent’s PSR model to enhance learning efficiency and facilitate utilization in large-scale multi-agent environments. Each agent undergoes independent DRL training and execution while parameters sharing is adopted. Experiments are conducted on real-world road networks and a large-scale road network comprising 1000 intersections.