Semi and Self-supervised Learning for Multi-label Image Classification: An Underwater Inspection Imagery Application

Underwater inspections are essential for the preventive maintenance of offshore equipment but can be costly and time-consuming, requiring specialized professionals. In this scenario, we propose a solution that helps inspection specialists find key events/objects in video frames by applying image classification models. In order to validate this approach, we use frames of real inspections from an oil and gas company. Some of this application’s challenges are inherent to underwater image classification, such as uneven lighting conditions, low contrast, and distortion. In addition, we have two significant issues induced by human labeling: label scarcity and noise. Due to the complexity of the assignment, traditional supervised models might fulfill the task. We address this problem by applying DINO ("Self-DIstillation with NO labels") and a new multi-label version of PAWS ("Predicting view Assignments With Support samples"), mPAWS ("multi-label PAWS"). The results show the benefits of using such models, with an improvement in the F1-Score of 2.7% compared to state-of-the-art supervised models. The models can perform real-time inference once deployed in the finished product, which has the potential to accelerate the underwater inspection procedure, making the work of the specialists more time-efficient and less prone to human errors.