A Regret-Informed Evolutionary Approach for Generating Adversarial Scenarios for Black-Box Off-Road Autonomy Systems

Developing autonomous vehicles (AVs) that operate in diverse and demanding environments is a difficult challenge. Two fundamental tools that can accelerate this process are testing an AV in diverse simulated environments and identifying core system weaknesses. While most efforts focus on improving these tools for on-road AVs, this letter focuses on an analogous set of tools for off-road AVs. A method called
Black-Box Adversarially Compounding Regret Through Evolution
(BACRE) is proposed for identifying adversarial scenarios using an evolutionary algorithm guided by a novel regret-based metric for general navigation tasks. A black-box approach is often preferable when system complexity can be diverse, like with off-road AVs, and when whole-system testing is required. A custom simulation platform is also provided to assist with the automated testing of AVs in diverse, unstructured environments. Numerical experiments demonstrate that BACRE's evolutionary process gradually increases scenario complexity to degrade vehicle performance (an effective and explainable process that comparable methods cannot achieve). Consequently, BACRE can streamline AV development by finding weaknesses at any development stage.