Increasingly Specialized Perception Network for Fine-Grained Visual Categorization of Butterfly Specimens

Existing fine-grained categorization methods predominantly conquer challenges independently, while neglecting the fact that patch proposal and feature extraction can reinforce each other. This necessitates to extract the domain-specific representations and localize key (most discriminative) patches alternately, since implicit to fine-grained specialization is the existence of an entry-category visual shared among all categories. In this study, an increasingly specialized perception convolutional neural network (ISP-CNN) is proposed, focusing on a butterfly domain at sub-species level due to the biosystematics structure. Its pipeline is an coarse-to-fine specialization that hierarchically extracts fine-scale features and proposes distinctive patches at multiple scales. Specifically, the framework consists of two highlights, i.e., hierarchical learning support vector machines (HL-SVMs) and patch proposal sub-networks (PPNs). Depending on the confidences obtained in HL-SVMs, the samples are classified at appropriate subset (i.e., sub-family, genus, and sub-species). Then the PPNs zoom the images to shift the focus on the most representational patches by taking previous predictions of HL-SVMs as a reference, while a finer scale network takes as input an amplified attended region from previous scales with gradual steps. As for self-optimization, ISP-CNN is driven by a patch-level loss and a class-level loss, to mutually learn patch proposals and decisions. For effectness verification, a total of 19,368 lab-made images of butterfly specimens spanning 48 sub-species are utilized as testing samples, while 116,208 augmented images are employed for training. ISP-CNN delivers the better or comparable performance, i.e., validation accuracy achieves 93.67% and testing accuracy achieves 92.13%, which outperforms state-of-the-arts.