BlackIceNet: Explainable AI-Enhanced Multimodal for Black Ice Detection to Prevent Accidents in Intelligent Vehicles

The advancement of intelligent transport systems and the rise of autonomous vehicles offer significant potential for reducing road accidents. However, mountainous regions, such as South Korea, are particularly susceptible to the formation of black ice, which poses a serious risk to both human and autonomous drivers due to its near-invisibility and sudden formation. Traditional road condition monitoring methods often fail to promptly detect black ice, underscoring the need for more advanced sensing systems. This work presents a multimodal system called BlackIceNet, integrating visual, acoustic, and sensor data, including surface and ambient temperatures, to detect black ice. The system utilizes a convolutional neural network (CNN)-based framework for image and audio analysis, followed by data fusion techniques to combine the insights from each modality. The proposed algorithm includes the following steps: preprocessing and normalizing data, feature extraction from visual and acoustic data, multimodal fusion to combine vision, audio, and sensor data, and classification of road surface conditions using BlackIceNet. The dataset has been gathered over the years from various testbeds established across three distinct areas in South Korea, contributing to a comprehensive understanding of the area’s diverse conditions. The evaluation results demonstrate the efficacy of the proposed method, achieving a 97.54% accuracy rate in detecting black ice with a model size of 233.47 MB and a training time of 2293.92 s, offering a more compact and computationally efficient solution. This fusion-based approach overcomes the limitations of individual modalities, providing reliable and early warnings, thereby enhancing road safety in hazardous conditions.