Reinforcement Learning with Safe Action Generation for Autonomous Racing

Reinforcement learning (RL) has been successfully applied to tackle many complex decision-making tasks. However, a critical problem is the unsafe exploration when deploying classical model-free RL methods in safety-critical systems. In this paper, we focus on safe RL for autonomous racing, which is formulated as a multi-objective optimization problem (MOP). We propose a novel mechanism, named Safe Action Generation (SAG), to address the above challenge. The mechanism mainly consists of two modules: a risk prediction module and an action generation module. The risk prediction module monitors the state of the vehicle in real-time, and once the state is evaluated as risky, the action generation module is activated to prevent unsafe behavior by restricting the exploration of the RL agent. Extensive experiments in The Open Racing Car Simulator (TORCS) demonstrate that our approach can complete one lap at an average speed of 108 km/h, Compared to the baseline without the SAG mechanism, our approach reduces the instances of driving off the track by 97.7%. By using the proposed framework, the learned policy can improve vehicle safety and generalization performance.