Contrastive Learning Based Knowledge Distillation for Enhancing Defect Detection

Defect detection is critical for maintaining industrial production quality. This paper introduces pre-trained contrastive learning models to enhance defect detection. We integrate the simCLR model with two methods: fine-tuning and training from scratch. Fine-tuning adjusts pre-trained weights for defect detection tasks, yielding superior performance with reduced loss and higher Top-1 Accuracy. Training from scratch initializes model parameters specifically for anomaly detection, offering a tailored approach. The method employs the reverse knowledge distillation (RD) model with a simCLR-trained backbone, leveraging self-supervised and unsupervised learning exclusively on defect-free samples for practical implementation. Evaluation is conducted on the MVTec Anomaly Detection Dataset, encompassing 15 industrial product categories with high-resolution images. The experiment results demonstrate that fine-tuning achieves optimal performance, maintaining lower loss and higher Top-1 Accuracy.