Online model-free adaptive traffic signal controller or an isolated intersection

This paper presents an online model-free adaptive traffic signal controller for an isolated intersection. The problem is formulated as a Reinforcement Learning (RL) task. We base our solution on the Q-learning algorithm. In contrast with other studies in the field, we use the queue length rather than the average delay as a measure of performance. Also, the number of queuing vehicles in non-conflicting directions is aggregated to represent a state. Then, instead of predefined phase splits or direct switching, the duration of phases is updated by a small amount of time. Finally, we include the queue reduction and equilibrium terms in the equation of an immediate reward. The performance of the proposed method is compared with an optimal symmetric and far-from-optimal asymmetric fixed signal plans. The experimental results show that the proposed method performs almost as good as an optimal one.