Deep reinforcement learning based green wave speed guidance for human-driven connected vehicles at signalized intersections

Green wave speed guidance is able to reduce the vehicle delay caused by unnecessary starting and braking at signalized intersections and ease traffic congestion, especially with the development of vehicle-to-infrastructure communications. Several methods have been proposed for connected vehicles in simplified environment by frequent control, which cannot be applied in real traffic situations. In this paper, we propose a Deep Q-Network (DQN) based method to learn to select an efficiency-optimal speed change mode for human-driven connected vehicles passing through the multi-lane and multi-intersection road network mixed with non-connected vehicles and random pedestrians. In order to avoid frequent speed fluctuations, we convert the continuous speed change into a smooth speed profile control by introducing vehicle dynamics model to transform the action space of DQN model into a series of control commands of the controller. By specifying an effective reward function to minimize the travel time in this task, the proposed method can achieve an effective mapping between the state representation and the corresponding ideal action. Experimental results on single-vehicle guidance and multi-vehicles guidance scenarios show that the proposed method outperforms two benchmark methods, obtains higher improvement in several travel efficiency metrics and more adaptive performance for dynamic traffic situations.