Developing a Deep Learning-Based Embedded System to Detect Adverse Weather Conditions for Autonomous Vehicles

In recent years, the development of autonomous vehicles has achieved significant progress thanks to breakthroughs in technology. However, unusual weather conditions such as heavy rain, snowfall, or heavy fog can affect the decision-making process of autonomous vehicles, thereby reducing their operational responsiveness and significantly impacting safety and reliability levels. This paper aims to develop a deep learning-based embedded system to detect and classify weather conditions, enhancing the decision-making capabilities of autonomous vehicles, therefor increasing safety and adaptability. We will experimentally assess the performance of three Convolutional Neural Network (CNN) models: ResNet-50, EfficientNet-B0, and SqueezeNet, deployed on a Raspberry Pi 4. Key tasks include collecting and preprocessing data, training the models using transfer learning, converting the trained models to tflite format and quantizing them to float16, deploying them on an embedded platform and evaluating their performance. Experimental results show that the EfficientNet-B0 model is the most effective. After quantization, the EfficientNet-B0 model can detect adverse weather in real-time with an accuracy rate of 88% on a combined dataset from the DAWM2020 and MCWCD2018 image sets, with an average processing time of 0.196 seconds and a processing speed of 4.8 FPS when running on a Raspberry Pi 4.