SkeFormer: Skeletal Cues-aware Bone point Relationship Learning for Efficient FBIC via Transformers

How to identify endangered bird species in complex outdoor environments has attracted significant attention in the fields of computer vision and machine learning. Previous studies on fine-grained bird image classification (FBIC) face numerous challenges, such as environmental occlusions and arbitrary postures, which limit the accuracy and robustness of existing methods. To address these challenges and enable more reliable bird species identification in extreme outdoor conditions, we propose a novel skeletal cues-aware bone point relationship learning for efficient FBIC via Transformers (SkeFormer). To the best of our knowledge, this is the first time skeletal relationships have been introduced to the FBIC task. Our model introduces three key modules: the skeletal relationship mining (SRM) module, the multilevel feature generation (MFG) module, and the key feature selection (KFS) module. Specifically, in SRM, the model mines the skeletal relationships among different bird species. In MFG, multiscale information is aggregated by connecting features across multiple layers. The KFS module selects key immutable regions of birds based on the learned skeletal relationships. Extensive experiments on two benchmark datasets, CUB-200-2011 and NABirds, show that SkeFormer outperforms existing state-ofthe- art models. The code for SkeFormer will be publicly available.