Multi-Vehicles Decision-Making in Interactive Highway Exit: A Graph Reinforcement Learning Approach

In the research of driverless decision-making, most of the current research is aimed at following, changing lanes, overtaking, and other scenarios. In this paper, algorithms and reward functions are designed to solve decision-making problems in interactive environments. An interaction and stochastic high-speed exit scenario of human-driven vehicles and autonomous vehicles (AVs) is designed. The features of the graph are directly extracted from the surrounding environment information of AVs by using the graph convolutional neural network (GCN). The steering angle and longitudinal acceleration are output through the neural network module. In addition, an exponentially increasing reward function based on driving purpose, traffic efficiency, driving comfort, and safety is designed. Moreover, this paper uses two algorithms, Graph Convolutional Deep Q-learning Network (GCN-DQN) and Graph Convolutional Double Deep Q-learning Network (GCN-DDQN), to train the driverless decision model and compare them with each other. According to simulation results, by adjusting the weight value in the reward function, the system can realize different control targets. Meanwhile, the decision-making model trained by GCN-DDQN has better performance under strong interactive random scenes than the GCN-DQN method.