A Review of the Impact of Rain on Camera-Based Perception in Automated Driving Systems

Automated vehicles rely heavily on image data from visible spectrum cameras to perform a wide range of tasks from object detection, classification, and avoidance to path planning. The availability and reliability of these sensors in adverse weather is therefore of critical importance to the safe and continuous operation of an automated vehicle. This review paper presents a data communication-inspired Image Formation Framework that characterizes the data flow from object through channel to sensor, and subsequent processing of the data. This framework is used to explore the degree to which adverse weather conditions affect the cameras used in automated vehicles for sensing and perception. The effects of rain on each element of the model are reviewed. Furthermore, the prevalence of these rain-induced changes in publicly available open-source datasets is reviewed. The degree to which synthetic rain generation techniques can accurately capture these changes is also examined. Finally, this paper offers some suggestions on how future adverse weather automotive datasets should be collected.