Multispectral Scene Classification via Cross-Modal Knowledge Distillation

Scene classification is a fundamental task for numeral remote sensing (RS) applications, which aims to assign semantic labels to image patches. Although deep neural networks (DNNs) demonstrated unique strength in scene classification, their performances are still limited due to the lack of training samples in the RS field. Recent studies show that the performance of scene classification can be improved by taking advantage of the knowledge transferred from models pretrained on RGB images. However, the modalities’ differences between input images hinder the knowledge transfer across models, especially when the input of the models has distinct spectral bands. To tackle the challenges, we propose a cross-modal knowledge distillation framework to improve the performance of multispectral scene classification by transferring the prior knowledge from teacher models pretrained on RGB images to the student network with limited samples. Moreover, a teacher assistant (TA) network is introduced to further improve the classification performance by bridging the gap between the teacher and student networks. The proposed strategy is evaluated on models with multimodality inputs with distinct spectral bands and demonstrates superior performance compared to the state-of-the-art methods.