AF-DQN: A Large-Scale Decision-Making Method at Unsignalized Intersections with Safe Action Filter and Efficient Exploratory Training Strategy

Autonomous driving is an advanced field that attracts significant attention and engages numerous researchers. However, relying solely on a single autonomous vehicle (AV) is insufficient to meet the demand of future transportation systems. This necessitates the application of connected and autonomous vehicles (CAVs), whose operation relies on multiagent decision-making technology. Currently, research primarily focuses on simple traffic scenarios. However, unsignalized intersections are frequently encountered in rural areas, characterized by high traffic volume, complex interactions, and significant risks. It is crucial to conduct research on the decision-making of CAVs at unsignalized intersections. To address these issues, the lane-changing decision-making of large-scale AVs at unsignalized intersections is studied in this paper. First, an action filter-based deep Q-network method named AF-DQN is proposed, which enables AVs to effectively filter out potentially hazardous lane-changing actions and execute safe actions. Additionally, a multi-objective reward function that considers multiple factors has been designed, including safety, task achievement, and compliance. Moreover, an exploratory training strategy is introduced to train the multi-agent deep reinforcement learning network model. The strategy facilitates agents to learn through exploration in simple scenarios before solving complex driving tasks in more complex scenarios. Finally, experiments are conducted to validate the effectiveness and superiority of the proposed method. Results show that exploratory training accelerates the model’s training speed and improves training effectiveness. Moreover, the AF-DQN method outperforms the baseline method in terms of safety, efficiency, and adherence to traffic rules.