Cross Modality Knowledge Distillation for Robust Pedestrian Detection in Low Light and Adverse Weather Conditions

RGB-based pedestrian detection is a challenging task in low light and adverse weather conditions because image quality can degrade rather significantly. Including other modalities such as thermal and gated imaging sensors can, on the other hand, significantly improve the detection performance in these conditions. However, these sensors are expensive, and including them may cause design and manufacturing challenges. In this paper, we propose a new framework that utilizes Cross Modality Knowledge Distillation (CMKD) to improve the performance of RGB-only pedestrian detection in low light and adverse weather conditions. Specifically, we develop two CMKD methods that rely on feature-based knowledge distillation and adversarial training to transfer knowledge from a pedestrian detector (teacher) that is trained using multiple modalities to a single modality detector (student) that is trained using RGB images only. Experimental results using the "Seeing Through Fog" dataset show that both of our proposed methods outperform the baseline detector in terms of detection accuracy without increasing computational complexity during inference. In particular, the proposed methods reduce the performance gap between teacher and baseline models by up to 55%.