A Validation Methodology for the Minimization of Unknown Unknowns in Autonomous Vehicle Systems

Deployment of SAE Level 3+ automated vehicles faces validation and certification challenges due to uncertainty and state space size of the operating domain. We propose a validation and testing methodology that aims to minimize unknown unknowns through minimization of scenarios that have not been accounted for, and scenarios that have not been identified due to modeling deficiencies. The methodology utilizes simulators with different levels of fidelity for residual risk handling, functional hierarchies for simplification of complex navigation tasks, and the Backtracking Process Algorithm to identify scenarios of risk significance. The methodology is demonstrated on a scenario with an intersection preceded by a traffic light. Through use of the testing flowchart, we were able to identify and remedy scenarios leading to undesirable events.