Fault injection in optical path - detection quality degradation analysis with Wasserstein distance

The goal of this paper is to present results of analysis of artificially generated disturbances imitating real defects of camera that occurs in the process of testing autonomous vehicles both during rides and later, in vehicle software simulation and their impact on quality of object detection. We focus on one perception module responsible for detection of other moving vehicles on the road. Injected faults are obliteration by particles and pulsating blinding. At the same time, we want to propose an examination approach scheme that will provide detailed information about distribution of quality in this comparative experiment. The method can be reused for different perception modules, faults, scene sets and also in order to compare new releases of main recognition software. To do so we combine statistical methods (Welch's ANOVA) and topological analysis (Clusterization over space of distributions, Wasserstein metric). Work provides a summary of the experiment for all data used and described by mentioned tools and examples of certain cases that illustrate general conclusions.