Optimizing Plankton Image Classification With Metadata-Enhanced Representation Learning

Automated camera-based sensors are widely used in vessel-based research to monitor plankton and marine particles. However, current methods suffer from the costly and time-consuming requirement of annotating data for fully supervised learning, especially in plankton grouping tasks characterized by long-tailed datasets. In response, we propose a novel self-supervised learning framework that significantly reduces reliance on expensive human annotations by leveraging crucial metadata such as water depth and location. The method comprises three major steps: self-supervised training, innovative sampling, and final classification. It identifies key sample subsets from an unlabeled dataset using a hierarchical clustering approach and incorporates an innovative balancing representative subsampling strategy that addresses the challenge of dataset imbalance and enhances generalizability across diverse plankton classes. Our approach prioritizes discerning representation features observed in images that exhibit correlations with the patterns found in their associated metadata. Furthermore, our method introduces a novel grouping based on the visual perspective selection method, enabling the identification of balanced subset views that depart from traditional class-based categorization. Our experimental results showcase a significant enhancement in image classification accuracy, with a 23% improvement over methods that do not utilize metadata, and attains a macro F1-score of 54% for ten populated species from a severely long-tailed dataset. This is achieved with a mere 0.3% of the entire dataset used for annotation.