Data-driven Optimal Traffic Signal Control with Phase Priority and Switching Cost

In this paper, a model-free scheme is proposed to solve the optimal traffic signal control of multi-intersection system. First, the model of the traffic network is constructed as a switching system with multiple subsystems. Each subsystem corresponds to a different traffic light phase. In addition, the reward function is designed by considering the priority of signal phase and the switching cost. Then, the problem is transformed to solve the corresponding Hamilton-Jacobi-Bellman (HJB) equation. The adaptive dynamic programming approach is utilized to find the optimal solution without information of system dynamics. Moreover, theoretical analysis of the proposed algorithm is presented including the continuity of value function w.r.t system state and the convergence of the iterated Q-function. For implementing the algorithm, the critic-only structure is employed which approximates the Q-function at each iteration. Finally, simulation results based on synthetic traffic data are provided to demonstrate the effectiveness of the data-driven method.