Automatic Detection and Classification of Defective Areas on Metal Parts by Using Adaptive Fusion of Faster R-CNN and Shape From Shading

Computer vision and deep learning approaches have an important role in industrial inspection systems. Computer vision technology is essential for fast, defect-free control of products in the production line. The importance of the computer vision concept is recognized when the problems of the classical methods are taken into consideration. Metallic defect detection is a challenging problem as metal surfaces are easily affected by environmental factors such as lighting and light reflection. Since traditional detection algorithms are inefficient in complex problems, we propose a novel method to detect and classify metal surface defects, such as cracks, scratches, inclusion, etc. The type and location of defects were detected by the Faster Regional Convolutional Neural Network (Faster R-CNN), combined with the Shape From Shading (SFS) method, which can extract surface characteristics. The Northeastern University (NEU) surface defect database was used for defective samples. The proposed algorithm has also been tested on an unlabeled dataset (KolektorSDD2/KSDD2) to show labeling performance. The results on both labeled and unlabeled datasets have demonstrated state-of-the-art performance in automatic defect detection, classification, and labeling. The proposed method has satisfactory results for the detection of defects on the metal surface, and the mean average precision is 0.83. The average precision of crazing, pitted surface, patches, scratches, inclusion, and rolled-in scale are 0.98, 0.81, 0,90, 0.79, 0.88, and 0.62, respectively.