Accelerated Verification of Autonomous Driving Systems based on Adaptive Subset Simulation

A throughout safety verification of an Autonomous Driving System (ADS) is essential to ensure its capability to drive safely in naturalistic traffic environment. Commonly used test automation methods such as Important Sampling (IS) do not perform well in distribution fitting efficiency in high-dimensional scenarios. In this paper, we proposed the Adaptive Subset Simulation (ADSS) method for safety verification for ADS. Two different systems under test were adopted: Baidu Apollo and Intelligent Driver Model (IDM). ADSS finds scenarios that are more and more critical iteratively and is capable to derive the accurate collision rate in naturalistic driving environment with fewer testing resources. A 3-dimensional car-following scenario and a 6-dimensional cut-in scenario were constructed in LGSVL to evaluate the performance of ADSS. The results showed that as the number of scenario dimensions increases, ADSS exhibits a more significant advantage over IS and vanilla Subset Simulation (SS) in terms of result accuracy and acceleration efficiency. In the 6-dimensional cut-in scenario, the efficiency of ADSS was 112 times faster than that of IS and 2 times faster than that of SS. Notably, ADSS demonstrated excellent testing effectiveness for both the complex ADS Apollo and the simple ADS IDM. These findings highlight the significance of utilizing ADSS in the safety verification of ADS.