Bi-Temporal Feature Relational Distillation for On-Board Lightweight Change Detection in Remote Sensing Imagery

Deep learning models for remote sensing change detection (CD) require significant resources, challenging real-time applications on spaceborne devices. Knowledge distillation (KD) technology is a potential solution that balances model size and CD performance. However, existing KD methods struggle to effectively transfer the teacher's ability to respond to changed landcovers, and also face challenges in assisting the student model in detecting changes in landcovers with subtle visual characteristics. To overcome these limitations, a bitemporal feature relational distillation (FRD) framework is proposed. The FRD framework includes two designed distillation components: bitemporal feature distance distillation (BFDD) and contrastive cluster representation distillation (CCRD). BFDD guides the student model to learn the relationship between bitemporal feature categories from the teacher model and introduces the iteration-wise random selection and bilinear interpolation strategy to solve feature mismatch problem between the teacher and the student. CCRD describes the pixel-level and cluster-level feature distributions through the designed contrastive cluster module, guiding the student model to align the change/nonchange features with the teacher model, so as to alleviate the confusion of change/nonchange features. Extensive experiments and analyses have confirmed that the proposed FRD framework can develop lightweight models capable of achieving performance comparable to large models, thereby enhancing suitability for real-time CD.