Towards Explainable Visual Vessel Recognition Using Fine-Grained Classification and Image Retrieval

The precise recognition of vessel types is critical for applications in maritime surveillance, but manual visual inspection is slow and error-prone. Automated fine-grained object recognition helps to quickly and accurately categorize vessels, as long as they overcome challenges of modern deep and machine learning-based methods such as class imbalance or intransparency. This paper utilizes recent literature to work towards this task considering two different problem formulations: fine-grained classification and image retrieval. We create a novel dataset called Military MARVEL consisting of 15,858 images of 137 military vessel classes to conduct the extensive, survey-like experiments. On two maritime datasets, we demonstrate that end-to-end fine-grained classification leads to slightly higher accuracy for the price of less flexibility compared to image retrieval. Heatmap-based eXplainable Artificial Intelligence methods are combined with the vessel recognition approaches to assess the achieved transparency quantitatively and qualitatively. Notably, quantitative measures are consistent with qualitative evaluation, especially in fine-grained classification. Our Military MARVEL dataset is available at https://github.com/HensoldtOptronicsCV/FineGrainedVesselRecognition.