Scene Text Detection in Foggy Weather Utilizing Knowledge Distillation of Diffusion Models

Adverse weather conditions can significantly hinder the performance of deep learning-based object detection models. Traditional approaches often rely on image restoration techniques to enhance the quality of degraded images prior to detection. However, these methods frequently struggle to balance image enhancement and detection tasks effectively, often overlooking latent information that could be beneficial for detection. To address these challenges, we propose a novel framework: Knowledge Distillation based on Diffusion Models (KDDM). This framework incorporates a Dehaze Network (DN), which employs large kernel convolution to remove weather-specific artifacts, thereby revealing more latent information. The DN, together with a text detector, forms an end-to-end scene text detection network, acting as the student network. Additionally, the nuanced internal representations of text-to-image diffusion models adeptly capture and integrate higher-order visual semantic concepts. Given the rich textual and visual content inherent in scene text, there is a fundamental connection to text-to-image diffusion models. As such, we utilize diffusion models as a teacher network to distill high-level visual semantic knowledge into the student network. Notably, we introduce an innovative distillation technique using a “Threshold_Mask”, which ensures that the student network focuses on text regions while minimizing interference from irrelevant background elements. Comprehensive experimental evaluations demonstrate that our KDDM framework significantly outperforms baseline models under foggy weather conditions, marking a substantial advancement in the field.