HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification

Fine-grained visual classification (FGVC) intends to confirm the sub-classes of a specific object category, e.g., identifying the species of dogs or birds. It is a challenging problem with the inter-class similarity among these sub-categories and intra-class variance in every fine-grained class. Most of the recent works intend to learn discriminative representations and class-consistency features. However, they only take the finest labels into account. We argue that the hierarchical label structure (HLS) implied in the category names can enhance the FGVC task. In this paper, we proposed two modules to leverage the hierarchical label structure. (i) We build a weighted graph in each batch based on the hierarchical label structure, the nodes of which are image features. The messages are passed among graph nodes for feature interaction. (ii) A hierarchy-aware ranking loss is proposed to regularize the distribution in feature space. The ablation study and experimental results show that our proposed modules achieve significant improvements over previous works.