Continually Evolved Feature and Classifiers Learning for Long-Tailed Class-Incremental Remote Sensing Scene Classification

Remote sensing data from real-world scenarios manifests a long-tailed distribution, with the continuous emergence of new classes over time. Nevertheless, the existing class-incremental remote sensing classification models neglect the above long-tailed distribution phenomenon, which seriously damages their overall superior performance. Meanwhile, long-tail class-incremental learning (LT-CIL) developed in other areas focuses only on the classifier decision boundary optimization of the tail class, while neglecting the robustness of the feature backbone. The feature backbone trained on the base classes causes a serious significant distribution shift for the incremental classes due to the distributional differences between base and incremental classes. To solve these issues, we propose a continually evolved feature and classifiers learning (CEF-CL) framework for long-tailed class-incremental remote sensing scene classification (LT-CIRSSC). Specifically, tail-class data are scaled and grafted onto head-class data to diversify the semantic information of the tail-class leveraging the rich context of the head classes, which can improve the generalization of the feature backbone. And then, an adaptive multiscale feature fusion (AMFF) module is proposed to couple feature maps of head- and tail-class scale-by-scale for generating virtual tail-class features that deeply perceive head-class information, which can further enhance the reliability of classifier decision boundary optimization. Furthermore, examples from old classes are regarded as pseudo-tail classes to participate in incremental learning, which greatly alleviates catastrophic forgetting of old classes. Extensive experiments on two remote sensing benchmarks demonstrate the superiority of the proposed CEF-CL in comparison with the existing class-incremental learning (CIL).