Automation of Traffic Control Systems with Deep Q-Learning Network

Traffic signal infrastructure is used to manage the flow of traffic and vehicle congestion that occurs on the road. In cities with high population densities and smart cities, traffic has been extremely hard to control, and vehicle delay times have been immense. High congestion due to traffic has caused an increase in the delay times of passengers and the need for an effective traffic control system. Thus, there is a need for automated traffic control systems. Tampering and altering the signals may cause traffic jams and can cause a delay due to signals displaying an incorrect phase for a particular intersection or junction. By implementing a Deep Q-Learning (DQN) network, the traffic signals can be automated with more efficiency. This algorithm will take into account the cumulative number of vehicles at the signal, and the vehicles' delay time and identify which signal phase to output. Instead of waiting for a full phase cycle to switch signals, the signals can change phases reactively to real-world traffic. Data about the vehicle's presence, vehicle count, and delay time are collected to predict the correct phase of a given signal. The significance of an attack can be quantified by a delay in travel time caused to a vehicle on the road. The model will take the vehicles' presence in each lane of the road from SUMO and identify which signal phase to produce as output, so the cumulative vehicle time can be minimized, i.e. delay time of vehicles at any instance is reduced. The machine learning model will predict which phase to change to, and the reinforcement learning algorithm will determine if the output has resulted in a desirable condition and provide arbitrary rewards as output. Implementing the DQN algorithm in place of the round-robin algorithm has reduced the cumulative vehicle delay time which is the time taken to reach a given destination and vehicle queue length.