IIIM-SAM: Zero-Shot Texture Anomaly Detection Without External Prompts

Anomaly detection is a crucial aspect in ensuring the reliability of industrial products in smart manufacturing. Despite employing unsupervised anomaly detection methods to mitigate the challenges associated with obtaining defect data, there remains the challenge of adapting anomaly detectors to drift within normal data distributions, especially when there are significant intra-class variations in normal samples. Consequently, zero-shot anomaly detection techniques, which do not require training data and thus avoid interference from training data, have emerged as a new research direction. Meanwhile, the zero-shot segmentation capability demonstrated by the Segment Anything Mode (SAM) visual foundation model has further fueled our research enthusiasm. In this paper, we focus on texture images and propose a no-external prompt SAM-based method for zero-shot texture anomaly detection, called Image Internal Information Mining SAM (IIIM-SAM). We utilize the Image Internal Information Mining Prompter to mine relationships between internal regions of the image, identifying specific background and potential defect points. The two types of prompt points are fed into the SAM model in different stages, imparting the SAM with incorporating category information segmentation capability. Unlike current methods, our zero-shot texture anomaly detection method requires no external prompts or expert knowledge, and also avoids the uncertainty introduced by text prompts necessary for CLIP-based approaches. Our experimental results demonstrate the superiority of our zero-shot texture anomaly detection method compared to other approaches. Specifically, in the texture subset of MVTecAD/KolektorSDD2, our IIIM-SAM achieves 99.2%/93.6% image-level AUROC and 98.6%/92.3% pixel-level AUROC. Note to Practitioners—The motivation of this paper arises from a practical challenge in anomaly detection in smart flexible manufacturing—customized customer demands lead to significant intra-class variations and even entirely different types of products. Currently, most anomaly detection methods struggle to adapt quickly to this challenge, severely impeding production efficiency. This paper proposes a zero-shot texture anomaly detection method based on SAM, which automatically generates background and potential defect points through an image internal information mining prompter. Compared to existing zero-shot anomaly detection algorithms based on foundation models, our method’s most significant uniqueness is without any expert knowledge and the absence of any external text prompts. Through comparisons with four advanced methods on six instances using two evaluation metrics, the results demonstrate the outstanding anomaly detection capability of our approach.