Solving the Vehicle Cooperation Problem at Signal-Free Intersection via an Asynchronous Deep Reinforcement Learning Approach

With the rapid development of modern intelligent transportation systems, connected and automated vehicles (CAVs) have garnered significant attention due to their advanced communication and decision-making capabilities. Intelligent collaborative decision-making in dynamic traffic scenarios poses significant challenges in the research of CAV technology. Strategies and methods have been developed to tackle the collaboration problem toward different scenarios. However, most existing studies have not been fully investigated the performance optimization and practicability of the vehicle collaboration at signal-free intersections. Meanwhile, the urban signal-free intersections represent a critical application scenario for vehicle cooperation, where a thorough study has not been given yet. Therefore, this study intends to solve the vehicle collaboration problem utilizing the deep reinforcement learning approach. Initially, the problem is formulated as an elaborated Markov decision process, comprising the state space, the action space, and the reward function. Then, a shared advantage actor-critic (A2C) model is proposed to effectively extract temporal and spatial features through the shared network, thereby improving the consistency of feature learning processes between the Actor and Critic networks. Furthermore, the asynchronous training strategy employed in this study involves multiple training processes concurrently, thereby enhancing the model’s convergence speed and stability. Finally, the effectiveness of the proposed method is verified in two typical intersection scenarios. Simulation results reveal that our method exhibits competitive performance compared with existing approaches, and an improvement up to 30% and 40% can be achieved in the retreat time and the averaged delay time. Additionally, the field experiments have been conducted on a miniaturized autonomous driving platform, verifying the considerable potential of the proposed method for real-world applications.