Safety-Guaranteed Policy Composition via Generalized Policy Improvement for Autonomous Vehicles

Autonomous driving has the potential to revolutionize transportation by improving safety, efficiency, and accessibility. However, existing methods, such as reinforcement learning (RL) approaches and rule-based strategies, struggle to achieve high sample efficiency, superior performance, and safety assurance, a challenge we refer to as the "impossible triangle". In this paper, we propose Safe-GPI, a novel decision-making method designed to address this issue. Safe-GPI employs Generalized Policy Improvement (GPI) to effectively combine multiple policies, thereby improving safety without compromising performance. Our theoretical analysis demonstrates that Safe-GPI maintains formal guarantees for collision avoidance under specific reward structures, while also providing performance assurances. Experimental results demonstrate that the simple combination of a rule-based model and a planning-based policy has the ability to significantly reduce collision rates, while achieving performance comparable to deep reinforcement learning methods. These findings suggest that Safe-GPI offers a feasible solution to the "impossible triangle", enhancing the safety and reliability of autonomous driving systems.