Analysis of Automotive Camera Sensor Noise Factors and Impact on Object Detection

Assisted and automated driving functions are increasingly deployed to support improved safety and efficiency and enhance the driver experience. However, there are still key technical challenges that need to be overcome, such as the degradation of perception sensor data due to noise factors. The quality of data being generated by sensors can directly impact the planning and control of the vehicle, which can affect the vehicle safety. This work builds on a recently proposed framework, analyzing noise factors on automotive light detection and ranging (LiDAR) sensors, and deploys it to camera sensors, focusing on the specific disturbed sensor outputs via a detailed analysis and classification of automotive camera-specific noise sources (30 noise factors are identified and classified in this work). Moreover, the noise factor analysis has identified two omnipresent and independent noise factors (i.e., obstruction and windshield distortion). These noise factors have been modeled to generate noisy camera data; their impact on the perception step, based on deep neural networks (NNs), has been evaluated when the noise factors are applied independently and simultaneously. It is demonstrated that the performance degradation from the combination of noise factors is not simply the accumulated performance degradation from each single factor, which raises the importance of including the simultaneous analysis of multiple noise factors. Thus, the framework can support and enhance the use of simulation for the development and testing of automated vehicles through careful consideration of the noise factors affecting camera data.