SALVO: Automated Generation of Diversified Tests for Self-driving Cars from Existing Maps

Simulation-based tests are more cost-effective and less dangerous than field tests; hence, they are becoming the norm for thoroughly testing self-driving cars in virtual environments. High-quality simulation-based testing requires physically accurate computer simulations, detailed and photo-realistic maps, and systematic approaches for generating tests. Moreover, since creating detailed maps is a manual process, they are expensive, and testers should avoid wasting such valuable resources, for example, by generating irrelevant test cases. To address those issues, we propose SALVO a fully automated approach to identify quantifiably diverse and critical driving scenarios that can be instantiated on existing high-definition maps and implement them in an industrial driving simulator as executable test cases. The evaluation of SALVO in the context of the 2021 IEEE Autonomous Driving AI Test Challenge showed that it could analyze maps of different complexity and identify many critical driving scenarios in minutes. Furthermore, the tests SALVO generated stressed a state-of-art self-driving car software in quantifiably diverse ways and exposed issues with its implementation.