An Inexactly Supervised Methodology Based on Multiple Instance Learning, Convolutional Neural Networks, and Dissimilarities for Interpretable Defect Detection and Localization on Textured Surfaces

The detection, localization, and interpretation of defects in textured surfaces pose challenges for automatic visual inspection. Both fully-supervised and weakly-supervised approaches have been proposed, where fully-supervised methods yield good results but require complex region proposal processes and labeled datasets, whereas with weakly-supervised methods, inexact labels that are less informative than fully labeled data are available. This paper introduces an alternative inexactly supervised methodology that performs defect detection and localization, along with a novel graphical interpretation of detected defects in textured surface images using image-level labels, without relying on region proposal algorithms or explicit defect annotations. The methodology employs block decomposition and bags as a representation using multiple instance learning, where feature vectors are generated from a convolutional neural network with transfer learning. Dissimilarities between bags are computed and the class label assignment is performed using a variant of the k-nearest neighbor algorithm. A baseline methodology using multiple instance learning and low-level feature extraction is also considered as a reference for comparison. The contribution of this study consists in providing a simple but powerful methodology in a way that graphically interpretable detection and localization results are obtained, enhancing the understanding of the detection outcomes. The proposed methodology is extensively evaluated using three datasets, one real and two synthetic, reporting various performance metrics and examples of localization and interpretation results. Average accuracies of 0.9722 ± 0.0058 and 0.9817 ± 0.0099 on synthetic datasets and a series of visualizations of the defect detection and localization results demonstrate the competitiveness of the proposal.