Autonomous Driving via Knowledge-Enhanced Safe Reinforcement Learning

Recently, the autonomous driving technology is at a critical phase evolving from typical, closed scenarios to largescale, open driving scenarios, which is challenged by the diversity and complexity of the driving environment. Reinforcement learning has the potential to address this challenge, but its application has been hindered by safety issues. To address the safety concern, we propose a Lagrangian framework called Knowledge-Enhanced Safe Reinforcement Learning (KESRL), which integrates prior knowledge of model-based methods into the model-free safe RL framework. Firstly, a model-based constraint based on high-order control barrier function is derived to deduce the safety criteria for the control system. Secondly, we propose a model-free constraint which estimates the distribution of accumulated safety cost and restricts the worst-case safety cost value. Furthermore, the above constraints are unified into the proposed Lagrangian framework to equip the end-to-end autonomous driving system with safety constraints. Experimental results conducted in the CARLA simulator demonstrate the effectiveness of the proposed framework KESRL in decreasing the collision rate and enhancing the control safety compared with several state-of-the-art RL methods.