How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies

Autonomous driving has the potential to revolutionize mobility and is hence an active area of research. In practice, the behavior of autonomous vehicles must be acceptable, i.e., efficient, safe, and interpretable. While vanilla reinforcement learning (RL) finds performant behavioral strategies, they are often unsafe and uninterpretable. Safety is introduced through Safe RL approaches, but they still mostly remain un-interpretable as the learned behavior is jointly optimized for safety and performance without modeling them separately. Interpretable machine learning is rarely applied to RL. This work proposes SafeDQN, which allows making the behavior of autonomous vehicles safe and interpretable while still being efficient. SafeDQN offers an understandable, semantic trade-off between the expected risk and the utility of actions while being algorithmically transparent. We show that SafeDQN finds interpretable and safe driving policies for various scenarios and demonstrate how state-of-the-art saliency techniques can help assess risk and utility.

I. Introduction

Achieving sustainable mobility in the future requires new modes of transportation. Autonomous vehicles present a huge opportunity by directly relieving drivers and improving, e.g., ride-sharing services. However, despite recent advances in perception and behavioral planning, traffic participation in complex, real scenarios remains an unsolved problem. Moreover, safety constraints and the need to understand and assess driving behaviors for regulatory approval prevent the use of many modern machine learning models.