Automotive LiDAR Based Precipitation State Estimation Using Physics Informed Spatio-Temporal 3D Convolutional Neural Networks (PIST-CNN)

With the rise of driving automation, optical sensors like cameras and LiDAR are playing a crucial role in vehicle perception. However, these sensors face challenges from harsh environmental conditions, including extreme temperatures and weather effects like fog and precipitation, which degrade their performance due to particle scattering. Consequently, significant efforts are being made to understand, model, and mitigate these effects. In this work, we address the reverse research question and demonstrate that the precise present precipitation state in form of the particle size and velocity distribution can be predicted using degraded Automotive LiDAR measurements and spatio-temporal 3D convolutional neural networks. By that, this approach provides a cost-effective solution for characterizing precipitation with a commercial Flash LiDAR sensor, which can be implemented as a lightweight vehicle software feature to issue advanced driver warnings, adapt driving dynamics, or serve as a data quality measure for weighted heterogeneous sensor data fusion and adaptive filtering during data pre-processing.