REAL-SAP: Real-time Evidence Aware Liable Safety Assessment for Perception in Autonomous Driving

Self-evaluation and monitoring are critical components in autonomous driving applications, especially for safety purposes, and yet there is no systematic framework to estimate the learning-based perception system in real-time. This paper aims to provide a general strategy for estimating how reliable the perception results generated from the black-box neural networks are in real-time. The perception safety evaluation problem has been formulated in a probabilistic framework, and theoretical analysis suggests that the existing geofencing or rule-based safety checking is a simplified version of the proposed strategy. The offline testing knowledge and real-time measured evidence are encoded as conditional probabilities and priors in the Bayesian network. The confidence score of the neural networks is utilized as an auxiliary factor to regularize the perception safety evaluation. Simulation and experimental results demonstrate the effectiveness of the proposed safety evaluation for perception systems under virtual and real data in city driving. In addition, we show that the proposed method can generate the necessary warning signals to support downstream safety monitoring and fail-degraded system functionalities.