On-line Reinforcement Learning Control for Urban Traffic Signals

It is quit difficult to archive perfect effects by applying the traditional modeling and control methods to the urban traffic signal control system because of non-linearity, fuzzyness, self-organization and uncertainty in the system. The artificial intelligence technologies may offer a new way to resolve this problem. In allusion to characteristics of the traffic signal control system, this paper proposes an on-line control algorithm based on Dyna-Q reinforcement learning, and utilizes the experiential knowledge gained by the traffic signal control agent in the trial-error process to estimate the model, and then plans the actions in the estimated model, accordingly it can accelerate the iterative process of the Q-learning. This paper adapts TSIS(a microscopic traffic analysis software) to implement the simulation on two traffic trunk roads which consist of 10 intersections. Comparing with fixed-time control, genetic algorithm and Q-learning control algorithm, simulation results indicate that Dyna-Q reinforcement learning algorithm has an obvious superiority.