Integrating Knowledge Distillation With Learning to Rank for Few-Shot Scene Classification

Few-shot learning (FSL) has great potential for automatic interpretation of remote sensing (RS) images. In this article, we make a study of few-shot RS scene classification. Many recently proposed FSL models adopt a very effective episode-based training procedure, where each episode is contrived to mimic the few-shot task by dividing training examples into support images and query images. However, these models can only judge whether or not a support image has the same class membership as a given query image. Such a yes-or-no prediction is rather rough, so we set a more demanding training objective, enforcing FSL models to rank support images according to support-query similarity and, hence, endowing them with better generalization ability. To this end, first, we propose ranking-preserving knowledge distillation (KD), which encourages a student network to rank support images in the same way as teacher networks. Then, integrating multiteacher KD with learning to rank, we construct a novel distillation loss using the Plackett–Luce distributions and build a novel few-shot classification model called ranking network. Extensive evaluation on two public RS datasets shows that the ranking network achieves the state of the art by a wide margin.