Research on Automatic Recognition of Casting Defects Based on Deep Learning

A method for recognition of casting defects based on improved You Only Look Once (YOLO v3) is proposed to address the problems of slow detection speed, low detection efficiency, and poor robustness suffered from the current inspecting manually methods, which can improve the ability to detect defects, especially for tiny defects. Firstly, for obtained the industrial digital radiography images (DR images), we introduce the guide filtering technique to enhance the defects in these DR images, thus obtaining standard defect samples; Further, the defect samples are annotated to generate the defect detection data set for network training. In this article, the improved YOLOv3 network model structure is used to detect defects. Comparative experiments illustrate the proposed defect detection model for castings achieves better performance. Concretely, the experimental results show that the improved network model (YOLOv3_134) converges faster than the YOLOv3 network model and has better convergence than the YOLOv3 model. And the mean average precision (mAP) of the YOLOv3_134 is 26.1% higher than that of the original YOLOv3, which makes the YOLOv3_134 model-based casting defect detection method meet the industrial production requirements in terms of accuracy and speed.


I. INTRODUCTION
Casting defects, such as blowholes, cracks, porosities and slag inclusions, not only affect the quality of the parts of the machine, but also reduce the work performance of the machine, which could cause serious traffic accidents and National property damage. Therefore, the defect detection plays an important role in casting manufacturing. Most traditional methods of casting defect detection are based on manual inspection, which is slow, inefficient and has poor robustness. Therefore, it is urgent to find a method that make the detection of casting defect efficient and accurate.
Traditionally, casting defect inspection is accomplished by human visual checking, which suffer from high labour cost and low efficiency. At present, most of the manufacturers are mainly using manual inspection for DR image defect detection of castings. The human visual checking is used to determine whether there are defects on the DR image of the castings, the type of defects, and whether it is to the extent that it needs to be overhauled or scrapped. The process has some The associate editor coordinating the review of this manuscript and approving it for publication was Inês Domingues . drawbacks as follows: (1) high labour cost and low efficiency; (2) low accuracy owing to the individual competence and subjectivity of the tester; (3) a large number of inspection tasks and huge inspection intensity.
In this article, we propose a deep learning-based method for the defect of castings detection in DR images, which can effectively avoid the occurrence of human mis-inspection and missed inspection and improve the accuracy of casting defect detection. Compared with manual detection methods, the deep learning-based defect detection methods has the following advantages: (1) lower manual labor and higher detection efficiency; (2) uniform and objective testing standards and higher stability; (3) less misdetections or missed detections when encountering a significant amount of the detection tasks.
At the same time, the development and application of deep learning in various fields has brought new ideas for defect detection in the manufacturing industry. Compared with traditional image processing technology and machine learning algorithms, the introduction of deep learning will enhance the ability of the casting DR image defect detection, thus improving the accuracy of casting DR image defect detection VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ and making it more capable of meeting actual production requirements. Therefore, the introduction of deep learning in this article is of great significance in the study of DR image defect detection of castings.
Currently, in direct defect detection methods based on image processing, Guo et al. [1] used the Maximum Between-Class Variance to achieve defect detection based on automatic thresholding of original images. Bai et al. [2] conducted the fourier transformation on the original images, resulting in effective enhancement of defective regions and attenuation suppression of defect-free regions of the image, which achieves more accurate detection of the location of casting defect. Guo et al. [3] proposed an automatic detection algorithm based on principal component analysis (PCA). Luo [4] proposed a defect detection algorithm based on hybrid SVM-QPSO to judge whether a defect exists or not. However, the traditional methods above for defect detection could only identify information such as the approximate location and size of the defect, lacking of the classification information of the defect.
Recently, the target detection based on deep learning has become popular in many industries, which can be applied to detect the location of the defects in casting and classify them.
Among deep learning-based defect detection methods, Liu et al. [5] established deep belief networks and trained it according to the source domain sample feature, but it needs to use transfer learning to extract feature information. Yu et al. [6] proposed a deep convolution neural network-based method to detect the casting defect in X-ray images, which is not effective in detecting small defects of casting. Jia et al. [7] proposed a three-dimensional localization method for casting defects based on deep learning feature matching. It can realize the automatic and accurate localization of the small defect in precise casting.
To further improve the accuracy of the defect detection, Ferguson et al. [8] showed the influence of defect detection of casting used varied feature extractor, such as VGG-16 and ResNet-101. Xu et al. [9] used feature cascade in VGG-16 to achieve better performance in defect detection. However, the methods above are not real-time, and cannot meet the balance between the speed and accuracy of defect detection. Moreover, the defects detection based on the methods above mainly address surface defect detection. To the best of our knowledge, there are no relevant research materials for the automatic detection of internal defects in castings.
The framework of target detection based on deep learning can be divided into two categories according to design principles, including region proposal-based framework such as the region-based convolutional neural networks (RCNN), Fast RCNN [10], Faster RCNN [11], Mask RCNN [12], and regression-based framework such as YOLO, single shot multibox detector (SSD), YOLOv2 and YOLOv3 [13]. The latter, the regression-based YOLOv3 algorithm, can ensure high recognition accuracy and faster detection speed, which is more suitable for the task of DR image defect detection of castings in this article. To validate the efficiency of our method, we compare our method with several state-of-the-art methods in terms of average precision (AP) and mean average precision (mAP) on our own dataset of castings [14]. Experimental results demonstrate that our method achieves better performance on mAP than compared methods.

II. CONSTRUCTION OF DEFECT DETECTION NETWORK IN CASTINGS
Based on the YOLOv3 network, we modify and extend the network model for detecting automatically the defects in castings. In the target detection network model, the design of the feature extraction network plays a key role.
YOLOv3 adopts an architecture, named as Darknet-53, to extract features, and the entire network is designed in fully convolutional manner. Each convolution component (DBL) of YOLOv3 is composed of convolution layer (Conv), batch normalization layer (BN) and activation layer (Leaky ReLU). And the residual component Resn is introduced to YOLOv3 network, which is beneficial to solve the problem of vanishing and exploding gradients and enhance the learning ability of the network. The overall structure of the YOLOv3 network is shown in Fig. 1.

A. INTRODUCTION OF DUAL-DENSITY CONVOLUTION LAYER STRUCTURE
As shown in Fig. 1, the three different scale feature maps are combined to detect the target. Due to the small size and different shapes of the defect in the DR images of the casting, we propose a method that incorporates dual-density convolutional layers into YOLOv3 to reduce the rate of missed flaw and improve the accuracy of the network.
The detection module of YOLOv3 network in each scale passes a 3 × 3 convolutional layer and a 1 × 1 convolutional layer before prediction. The former is to extract image features and increase the number of network channels. The latter is to change the size of the network channels. This article replaces the single density convolutional layer detection of YOLOv3 by dual-density convolution layer. The improved structure is shown in Fig. 2.
According to the structure of the dual-density convolutional layer in Fig. 2, the detection module is modified to a parallel dual-density layer model of a 3 × 3 convolutional layer and two series 3 × 3 convolutional layers. More detailed information is showed in the smaller receptive field of a 3 × 3 convolutional layer. Two series 3 × 3 convolutional layer  has larger receptive field and more semantic information that can detect large defects efficiently. Therefore, YOLOv3 network obtains two different receptive fields when extracting feature maps of various scales. Using more detailed information and semantic information, the network module improves the ability of the detection.

B. INCREASING SCALE OF MODULE PREDICTION
Three different scale feature maps are used in YOLOv3 for object detection. Different scale can detect different size of defect. The outputs of the feature maps are 1/32, 1/16 and 1/8 respectively.
To detect the small size of defects such as porosities and blowholes on DR images of castings, we expand three different scale feature maps in YOLOv3 to four. Through the research and analysis, we add 104 × 104 scale feature maps. The structure after enlarged the scale of module prediction is shown in Fig. 3.
The network structure has changed due to the introduction of dual-density convolution layer and the increasement of the scale of module prediction. The improved network model of defect detection in castings, YOLOv3_134, augments from 106 layers to 134 layers. We add dual-density convolution layer structure into 80th-84th layers, 96th-100th layers, 112th-116th layers and 128th-132th layers, and 19th-134th is prediction output of 104 × 104 feature maps. The memory cost of the traditional YOLOv3 network is 2006MB, and that of the improved network is 2284MB. In general, compared with the traditional methods, the memory cost of the YOLOv3_134 has increased slightly. In general, structure advancement in YOLOv3 including the introduction of dual-density convolution layer structure and the increasing scale of module prediction can enhance the accuracy of defects detection in castings, especially for tiny defects.

C. LOSS FUNCTION OF THE NETWORK
The loss function is used to judge the difference between the predicted value of the model and the ground-truth. For YOLOv3, its loss function demonstrates the gap between the predicted values and real values, which can be reduced by training the proposed model. The loss function of YOLOv3 is to reach the best balance on the three indicators when detecting and outputting: the coordinates, the categories and the confidence scores of the predicted target.
The loss function integrates the localization loss L loc (l, g), confidence loss L cla (O, C) and classification loss L conf (o, c) of the target using the method of mean-square error. The loss function is shown as in Equ (1). Among them, λ 1 , λ 2 , λ 3 are balance coefficients.

B. CONSTRUCTION OF CASTING DEFECT DATASETS 1) PREPROCESSING
Considering that the original DR images of the castings are foggy and have no prominent detailed information, we need to smooth the images. However, the consistency processing can be conducted to the information of the image noise and contour by the Isotropic filtering. Important textures and relevant details are filtered while the noise is filtered, which results in the loss of much defect information and interferes with the correct identification of defects. In this article, Guided Filter is used as an edge-preserving filter, which preserves the edge information to reduce the loss of important textures and related details.
The images and their grayscale distribution before and after pre-processing by guide-filtering are shown in Fig. 4 and Fig. 5. From the original images, we can see that there are blowhole defects, but the distribution of blowhole defects is scattered and the grayscale values of the defect areas are close to the background areas. If the original image is used as the image data set for defect detection of castings, it will affect the training effect of the model to a great extent. After oriented filtering of the image, we can find that the contrast between the defect area and the background area is more obvious after pre-processing, and the gray value histogram distribution is more uniform, which enhances the outline and detail of the defect area and also suppresses the increase of VOLUME 9, 2021  the gray value in the background area and facilitating the detection of casting defects.

2) DATASETS CONSTRUCTION
The datasets contain 932 DR images, each of which is three-channel and has a pixel size of 2240 * 2048. Every DR image may have blowholes and porosities. As shown in Fig. 6, according to the pixel and size, each defect is divided into 1-5 levels. (a)-(e) are blowholes, and (f)-(j) are porosities. After statistics, the defect categories in the training set are shown in the Table.1, and the defect categories in the test set are shown in the Table.2. When labeling them hierarchically using the LabelImg, we stipulate the label a represents the blowhole, and the label c represents the looseness. We format and store the labeled files as XML, and convert the XML file to a TXT file in < label, X, Y, W, H > format, which means the type and the grade of the defect contained in the rectangular box, the coordinates of the center point of the rectangular box, and the width and height of the rectangular box.

C. TRAINING
In our experiment, the momentum gradient descent method is used to train the casting defect DR image target detection model 40,000 iterations. We also used the Adam optimizer, but it cannot increase the accuracy obviously. During training,  the batch size is set to 64. The momentum factor is set to a fixed value of 0.9. And the initial learning rate is set to 0.001. In order to prevent overfitting, a strategy based on exponential decay function is used to reduce the learning rate to 0.0001 by stages. The batch size is set to 1 when testing. The ratio between the training set and the test set of DR image samples for casting defects is 9:1.
To test the validity of the improved method, the original YOLOv3 model and the improved model (YOLOv3_134) that detected the defects for casting DR image professionally were used separately. The comparative analysis of the experimental results is used to determine whether the improved model meets the requirements of real-time detection while ensuring accuracy of identification.

D. COMPARATIVE EXPERIMENTS AND ANALYSIS 1) COMPARATIVE EXPERIMENT BASED ON THE LOSS FUNCTION CURVE
The loss function curve reflects the update speed of the network model parameters, whether the network model has converged, and whether the training is completed. The more rapidly the loss function drops, the faster the network model parameters in the process of continuous learning update. when the loss function value change curve tends to stabilize, the model training is about to be completed, which means that the network model training is close to convergence.   This article compares the YOLOv3 network model and the YOLOv3_134 network model during the training process to analyze the loss value of the loss function as the number of iterations increases. The comparative curve is shown in Fig. 7.
It can be seen from Fig. 7 that as the number of iterations of the training process increases, the curves of the YOLOv3 and the YOLOv3_ 134 model loss function eventually smooth out and the training process of the model reaches the convergent state. As shown in the figure, the loss function value of YOLOv3_134 gradually tends to stabilize around 2.0 at 20000 iterations, but the YOLOv3 stabilize at around 3.5 after 30,000 iterations.
The analysis yields that the YOLOv3_134 model converges faster and more strongly than the YOLOv3 network model. It is yielded that the YOLOv3_134 model converges faster and more strongly than the YOLOv3 network model.
The loss function curve of the YOLOv3 is more volatile than that of the YOLOv3_134 model, which can show that the YOLOv3_134 model is more stable and has a better fit to the sample than YOLOv3 when training and learning on the dataset. Therefore, by analyzing the fast and slow decline of the loss function value change curve and the loss value that eventually tends to stabilize, it can be verified that the improved YOLOv3_134 model is effective in the detection of casting defects.

2) COMPARATIVE EXPERIMENT BASED ON SENSITIVITY OF SMALL TARGET DETECTION
Whether the improved model can detect small targets accurately is an important criterion for judging the model's ability of detecting defects that have varying morphology and small size on DR images of castings.
We can get the different weights obtained by the YOLOv3 and YOLOv3_134 during the training process to inspect the datasets. A comparative analysis of the detection results between improved and not improved model is performed to illustrate the sensitivity of small target defects. The detection results of the YOLOv3 and the YOLOv3_134 are shown in Fig. 8.
Through the comparative analysis of the results in Fig. 8, it is found that the YOLOv3 model-based defect detection method can identify the obvious defects easily in the DR VOLUME 9, 2021 image defect detection of castings, mainly the loose and blowhole defects of level 3 (a3, c3), level 4 (a4, c4) and level 5 (a5, c5), while that of level 1 (a1, c1) and level 2 (a2, c2) are difficult to be effectively detected.
The YOLOv3_134 can detect the information of the defect localization more accurately and the robustness and sensitivity of the small defects are improved overall. In addition, the comparative analysis of the detection results also  shows that the model based on YOLOv3_134 has fewer rate of false detection and better capabilities of the model characterization.
There is a conclusion that the identification of casting defects based on the YOLOv3_134 is significantly better than that based on the YOLOv3.

3) COMPARATIVE EXPERIMENT TO VALIDATE THE EFFECTIVENESS OF YOLOV3_134
To validate the effectiveness of the improved YOLOv3 to detect the defects, we analyzed and compared the YOLOv3 model (Model 1), the model of the improved initial anchor box clustering (Model 2), the model after introducing the double-density convolutional layer structure (Model 3), the model after adding the prediction scale (Model 4), and the improved YOLOv3_134 model (Model 5). The model evaluation indexes AP, mAP, and FPS are used to evaluate the advantages and disadvantages of each model and verify the effectiveness of the improvement strategy for DR image defect detection of castings. The results of are shown in Table.3.
As shown in Table. 3, the mean accuracy of the improved YOLOv3_134 model reached 88.02%. The FPS is 39 frames/s, which can meet the real-time inspection requirements as soon as improve the ability of defect identification.
As shown in Table. 4, the mAP increases by 5.8% after improving the initialization clustering algorithm of anchor box, and it increase 14.9% and 15.0% respectively after introducing dual-density convolution layer structure and adding prediction scale. It isn't doubt that the improvement of the anchor box initialization clustering algorithm, the introduction of double-density convolutional layer structure and the addition of the model prediction scale can greatly improve the accuracy of the model in detecting small defects.
Overall, the mAP of YOLOv3_134 improved by 26.1% compared to the YOLOv3, which means YOLOv3_134 can meet the industrial requirements in accuracy and speed.

4) RESULT ANALYSIS OF VARIED MODEL
We compared and analyzed the results of different target detection models in order to verify the validity of the improved YOLOv3_134 in application, included the YOLOv2, Faster R-CNN, YOLOv3 and YOLOv3_134. The results are shown in Table. 4.
As shown in Table. 4, comparing with the YOLOv2 and YOLOv3, the improved YOLOv3_134 has a higher accuracy of real-time detection, which indicates the effectiveness of the method. Compared with Faster R-CNN that is not real-time, although the YOLOv3_134 is slightly lower in accuracy, it is more suitable for industrial production of casting defect detection system.
In conclusion, the YOLOv3_134 can achieve the balance between the accuracy and speed at the same time and is more accurate than any other detection methods currently.

IV. CONCLUSION
Compared with the traditional image recognition techniques, the deep learning method solves the trouble of constructing VOLUME 9, 2021 features manually and are more transferable. In this article, the technique of graded detection of each defect in DR images of casting defects is not possible with most traditional image inspection techniques. After the improvement of the anchor box initialization clustering algorithm, the introduction of double-density convolutional layer structure and the addition of the model prediction scale, the YOLOv3 is improved to get a better network model for detecting casting defects. The results show that the mean average accuracy of casting defect detection is significantly improved compared with the original YOLOv3 and our method to detect the defects of casting is real-time compared with other target detection network, which means that the improved YOLOv3_134 network model can detect casting defects more quickly and accurately. Our method has been applied in industrial production.