Research on decision-making of lane-changing of automated vehicles in highway confluence area based on deep reinforcement learning

The traffic environment in the highway confluence area is complicated, and accidents are prone to occur when changing lanes, which is a major difficulty in highway vehicle driving. In view of the low adaptability of rule-based decision-making algorithms to environmental changes and the simplification of environmental complexity in existing learning-based decision-making algorithms, we had established Deep Q Network and Double Deep Q Network models for lane-changing decision of an automated vehicle in the confluence area based on deep reinforcement learning, as well as the autonomous lane-changing decision model for environmental vehicles, and two new reward functions were proposed to restore the complexity of the traffic environment to the greatest extent. In the constructed virtual traffic environment, the simulation of lane change in the confluence area was carried out, and the results showed that the newly added reward function increased the success rate of lane change by 4%. In the simulation of changes in the speed of the automated vehicle, changes in the number of environmental vehicles and changes in the driving style of environmental vehicles, the strong adaptability of lane-changing decision algorithm of Double Deep Q Network of the deep reinforcement learning was verified, and the lane-changing success rates were over 90%.