Applications of Distributional Soft Actor-Critic in Real-world Autonomous Driving

Reinforcement learning (RL) plays an important role in the decision-making of high-level autonomous vehicles due to the self-evolving ability without reliance on labeled data. Although existing RL-based decision-making studies have yielded fruitful results, most of them are carried out based on simulation platforms. Due to the inherent difference between simulation and the real world, it is of great significance to verify the efficacy of RL-based decision-making in practical applications. In this paper, a multi-lane driving task and the corresponding reward function are designed to provide a basis for RL-based policy learning. The distributional soft actor-critic algorithm is used to learn an offline policy based on a simulated environment. Then, we implement the learned policy to a real car on a two-lane park road. Both objective and subjective experiments are carried out to verify the effectiveness and robustness of the learned policy in practical applications. Experimental results show the trained policy can not only complete driving tasks smoothly and robustly, but also acquire fair satisfaction from subjects. Our work provides certain evidence for the feasibility of RL in real-world driving tasks.