EvoScenario: Integrating Road Structures into Critical Scenario Generation for Autonomous Driving System Testing

Autonomous Driving Systems (ADS) are safety-critical and require comprehensive testing before their deployment on public roads. Most existing testing approaches consist in generating scenarios that vary the behaviors of dynamic objects, while leaving a predefined road environment unchanged. Consequently, these approaches overlook the influence of different road structures on ADS safety, e.g., collisions can happen more frequently than usual on a merging road, because of the specific road structure. In this paper, we propose EvoScenario, a novel approach that integrates road structures into the generation of critical scenarios for exposing safety risks of ADS. Specifically, EvoScenario models a driving road as a sequence of road segments characterized in different aspects, such as their shapes and widths. Then, a test case is defined by concatenating the sequence of road segments and the sequence of dynamic object maneuvers. Inspired by EvoSuite that generates sequential method calls for Java unit testing, EvoScenario leverages the sequential models of test cases and constructs a multi-objective optimization framework to search for critical scenarios. We implement and demonstrate EvoScenario on an ADS provided by our industrial partner. Evaluation results show that EvoScenario can identify 6 types of safety violations, and outperform existing baseline testing approaches.