Reinforcement Learning-Based On-Ramp Merging Decision-Making for Autonomous Vehicles

On-ramp merging is a complex and high-accidents scenario. The complexity of on-ramp merging scenario is mainly reflected in the aspects of varied merging environment and uncertain driving behavior of the surrounding vehicles, so it is a challenging scenario for autonomous vehicles. In this paper, a deep Q-learning network (DQN)-based merging decision-making method is proposed for autonomous vehicles. First, the models of vehicle dynamics and on-ramp merging scenarios are built, and the merging motivation is quantitative for designing the reinforcement learning. Second, the actions of the autonomous vehicle are defined, and a value-based DQN network for the merging decision-making is designed by simultaneously considering the driving safety, efficiency and comfort. Finally, in order to evaluate the performance of DQN-based merging decision-making method, a dynamic programming (DP)-based merging decision-making method is selected as the benchmarked method. The validation results demonstrate the autonomous vehicle by the proposed DQN-based merging decision-making method possesses good performance in safety, efficiency and comfort at on-ramp merging scenarios.