Coordinated Traffic-Signal Control of Wide Area Network via Hierarchical Reinforcement Learning

Traffic-signal control is key to ensuring smooth traffic flows in urban areas. However, controlling traffic by considering various traffic characteristics is a complex and challenging task. Although rule-based methods are typically employed, they have limitations. In this context, deep reinforcement learning-based methods have attracted attention because they do not require environmental models and can generate control policies without human intervention. Previous studies on traffic signal control have primarily employed autonomous decentralized control using the information from each intersection and implementing coordination among intersections. However, optimizing traffic control for an entire city using only data from near the intersections is challenging. In addition, learning the primitive control rules of traffic signals using all the information across a wide area significantly increases the action dimension and further complicates learning. However, this is determined ad-hoc and does not guarantee rationality. Therefore, this paper presents a hierarchical architecture that divides global and local controls. By introducing hierarchy and applying multiobjective reinforcement learning, this study proposes switching multiple intersection-control policies corresponding to the traffic-flow ratio, which guarantees Pareto optimization of traffic signal control among intersections.