Adaptive Selecting and Learning Network and a New Benchmark for Imbalanced Fine-Grained Ship Classification

Ship classification is one of the most essential tasks in ship surveillance, which is an important but challenging problem. Most existing methods are designed for remote sensing images and there are few works processing natural ship images captured by camera. In this paper, we design a new framework called Adaptive Selecting and Learning Network (ASL), to solve the problem of fine-grained classification of ship natural images in the real world. Our method has two key contributions to enable more effective learning from the imbalanced ship data. First, we present a memory network equipped with an adaptive selecting learning strategy to selectively memorize the hard samples that are difficult to classify. The presented learning strategy can re-balance the data distribution of different classes in the training procedure, which achieves more effective learning. Second, we design an inference network with an attention mechanism, to capture the structural similarities between new samples and hard samples. The attention mechanism can enhance the training of hard samples and yields better learning performance. Moreover, our framework can be easily combined with existing fine-grained methods. In addition, we propose a new Dachan Island Ship (DIS) dataset, which is collected in the real-world scenarios. The DIS dataset has a significant imbalanced distribution between classes that aligns with the real situation. Extensive experimental results on the proposed DIS show that our model outperforms most of the existing fine-grained classification methods.