Industrial Camera-Based Powder Bed Defect Detection in Laser Powder Bed Fusion: Dataset Construction and Model Evaluation

Laser Powder Bed Fusion (LPBF) is a crucial additive manufacturing technique that builds complex geometries by selectively melting metal powders. However, it faces challenges from defects such as ditches, stacking, insufficient spreading, and craters, which can affect the mechanical properties and quality of the final product. To enhance manufacturing quality and stability, we developed a specialized dataset for detecting these defects, comprising 420 images of ditches, 800 of insufficient spreading, 657 of craters, and 600 of stacking. We evaluated 16 state-of-the-art deep learning models on this dataset and implemented data augmentation techniques, including rotation, scaling, and noise addition, to increase the dataset size fivefold to 12,385 images. Our results showed that data augmentation significantly improved model performance. The YOLOv10 series excelled in post-processing speed at 0.7 milliseconds, while YOLOv8-n had strong inference capabilities at 12.4 milliseconds. YOLOv9-t achieved the fastest preprocessing at 0.3 milliseconds, and YOLOv9-e attained the highest mAP@0.50 score, with precision and recall rates of 0.965 and 0.975, respectively. This study provides a high-quality dataset for powder bed defect detection and validates the efficacy of advanced deep learning models in this field.

SECTION I. Introduction

Additive Manufacturing (AM) has emerged as one of the most revolutionary manufacturing technologies, gaining widespread adoption and application over the past two decades [1]. Among the various AM techniques, LPBF is one of the most widely used metal additive manufacturing technologies [2]. It constructs parts by layer-by-layer deposition of metal powder, using laser energy to fuse the metal powder. The quality of parts manufactured using LPBF is influenced by numerous factors, with the powder laying process playing a critical role in determining both the workpiece quality and structural performance. The effectiveness of powder spreading directly impacts the production quality of components [3]. During the repetitive powder spreading process, defects such as insufficient spreading, ditch defects, and foreign particles pose significant challenges to the LPBF process. These defects can lead to uneven surfaces after the laser melting of the metal powder layer [4]. Severe powder bed defects can disrupt the printing process, leading to quality issues such as pits, cracks, or irregular surfaces within the part, and can also cause damage to the recoater, the LPBF build plate, and the elevating platform. Therefore, ensuring the quality of each powder bed layer in the LPBF process is crucial for improving part quality and final performance [5].

In recent years, Deep Learning (DL) technologies have shown significant potential in detecting and classifying powder bed defects during the LPBF process [6]. Scime and Beuth [7] utilized a Multi-scale Convolutional Neural Network (MsCNN) to detect and differentiate anomalies in the LPBF powder bed. Additionally, Scime et al. [8] proposed the Dynamic Segmentation Convolutional Neural Network (DSCNN) model, which was successfully demonstrated on data from nine different powder bed AM machines, covering three AM technologies, achieving semantic segmentation of powder bed imaging data. Shi and Chen [9] developed a powder bed detection system using an industrial camera and multiple lighting sources. By studying the characteristics of defects under different lighting conditions, they proposed improved lighting strategies. They employed image feature enhancement based on the grayscale characteristics of powder bed images and an adaptive threshold segmentation algorithm to isolate defect areas. Their study encompassed three Convolutional Neural Network (CNN) algorithms: AlexNet, ResNet-50, and VGG-16, which were experimentally compared and analyzed for stripe, ultra-high, and incomplete powder laying defects in the current powder layer.

The YOLO (You Only Look Once) [11] series of models has garnered significant attention for its speed and accuracy in object detection tasks. Zhao et al. [12] employed YOLOv5x, Faster R-CNN, and SSD models to detect three typical powder bed defects: ditches, stacking, and insufficient spreading. They utilized data augmentation techniques to expand the training images and employed Transfer Learning (TL) methods to train the models. Among these models, YOLOv5x exhibited superior performance compared to Faster R-CNN and SSD. Other researchers have utilized high-speed cameras, near-infrared cameras, and thermal imaging devices combined with deep learning algorithms for the identification and detection of defects during the powder spreading process [13], [14], [15]. Bevans et al. used features captured by splash imaging cameras, near-infrared tomography cameras, and optical cameras for powder bed imaging as inputs to simple machine learning models, jointly detecting porosity, layer-level distortion, and geometry-related flaws [16]. Gobert et al. proposed a defect detection method based on supervised learning and in-situ sensing technology for the powder bed fusion (PBF) process, enabling real-time detection and classification of defects during the build process [17]. Print time is a critical factor influencing the overall cost of AM [18] Optimizing the speed and efficiency of the production process is crucial, especially in the real-time detection of powder bed defects. In the literature, both single-stage and two-stage object detection algorithms have been used for LPBF defect detection and localization [12]. From an AM defect identification standpoint, the single-stage methods eliminate the need for region proposals, saving computational time. Experiments have shown that YOLO outperforms Faster R-CNN in precise bounding box prediction. Faster R-CNN, a two-stage detector, is significantly slower than YOLO and occasionally overfits or misses the most prominent objects [19]. Given YOLO’s small size, compact structure, good frame rates on real-time high-speed cameras, and low hardware requirements [20], it was selected for powder bed defect detection. YOLO offers significant advantages over other deep learning algorithms in this field. Its exceptional combination of speed and high accuracy makes it particularly well-suited for real-time applications requiring both rapid processing and high precision [21]. Although Transformer models achieve high pixel-level defect recognition accuracy, YOLO models have a clear advantage in terms of model parameter size and detection speed, which is particularly crucial for real-time industrial deployment applications [22]. This efficiency not only ensures the quality of the production process but also plays a crucial role in reducing production costs by minimizing delays and preventing defects.

The Layer-wise Imaging Dataset from Powder Bed Additive Manufacturing Processes provides comprehensive multi-modal data and detailed annotations across various technologies. However, the dataset includes training data with ground truth pixel masks that must be converted into a format suitable for YOLO, such as bounding boxes. Moreover, the extensive range of modalities and detailed annotations may require substantial computational resources for processing and model training, making it potentially unsuitable for use on resource-constrained devices [23]. Although YOLO models have been used to detect three typical types of powder bed defects [12], the application of the latest object detection models in powder bed defect detection is still limited, and comprehensive comparisons of multiple models in this context are lacking. Moreover, existing research often focuses on a narrow range of defect types, making comprehensive detection of all potential defect types a persistent challenge. Therefore, it is crucial to develop a comprehensive, well-annotated dataset that encompasses a wide range of powder bed defect types.

This study compares various state-of-the-art object detection algorithm models, achieving automatic and rapid classification and localization of multiple powder bed defects during the LPBF process using bounding boxes. Specifically, this study presents the following contributions: (1) Publicly Available Powder Bed Defect Dataset: We have released a detailed annotated dataset of powder bed defects, providing valuable resources for researchers in related fields and promoting the development and application of powder bed defect detection technology. (2) Comparison of Model Performance: We systematically studied and compared the performance of various deep learning models in powder bed defect detection, identifying the most suitable models for this type of defect detection and filling the gap in existing research regarding the lack of systematic comparison of different models’ performance. (3) Analysis of Data Augmentation and Training Data Volume: We conducted an in-depth analysis of the impact of data augmentation methods and training data volume on the detection accuracy of YOLO series models, providing specific strategies to optimize model performance, and offering guidance for subsequent research and practical applications. (4) Efficient Defect Classification and Localization: By leveraging YOLOv8, YOLOv9, and YOLOv10, this study aims to achieve the automated and rapid classification and localization of multiple powder bed defects during the LPBF process. Integrating this detection system into the production workflow enables real-time monitoring and control, significantly reducing the occurrence of defects and enhancing overall production efficiency.

Figure 1 presents the overall approach for real-time monitoring of the working area during laser additive manufacturing (e.g., selective laser melting), using an industrial camera in conjunction with YOLO-based deep learning detection networks (v8, v9, v10) to automatically identify and label surface defects. The schematic on the left illustrates the selective melting of the powder bed by the laser source, galvanometric scanner, and F-theta lens, enabling layer-by-layer part fabrication in a protective gas environment. The photograph on the right shows the actual placement of the industrial camera and examples of defect images captured from different angles (with red dashed boxes indicating defect areas), demonstrating the effectiveness of this method for the early detection and identification of quality issues in the manufacturing process. Furthermore, once the object detection network identifies defects in the powder bed, it can initiate a recoating process, which is crucial for the real-time detection and remediation of defects. This study primarily employs visible light cameras for powder bed defect detection, as they are cost-effective and widely used in optical imaging applications. In contrast, infrared imaging is typically utilized to monitor surface temperature distribution in parts [16], while ultrasonic testing is primarily used to detect porosity [19]. Given the advantages of low cost and efficiency, visible light cameras are the preferred choice for detecting powder bed defects, making optical imaging the most common approach for this task.

FIGURE 1.

Schematic of powder bed defect detection during laser additive manufacturing.

SECTION II. Methods and Data

This section provides a detailed analysis of the network architecture and working principles of three state-of-the-art YOLO algorithms: YOLOv8, YOLOv9, and YOLOv10. To ensure a fair comparison across different model configurations, a constant dataset was used for training. A total of 16 model configurations were evaluated: five configurations for YOLOv8, five for YOLOv9, and six for YOLOv10. These models differ in terms of parameters and network architecture. The differences in the number of parameters for each model [24] are illustrated in Figure 2. The COCO AP (%) denotes the mAP@0.5:0.95 metric measured on the 5000-image COCO val2017 dataset. The variations in network architecture will be introduced in the next section.

FIGURE 2.

Performance comparison of different yolo models on the coco object detection dataset.

Additionally, we provide a detailed description of the dataset preparation and data augmentation methods, as well as the evaluation metrics used to analyze the detection results. All model configurations were trained on the same computer system using identical hyperparameter settings to ensure consistency and comparability of the results. Figure 3 illustrates the specific methodology employed, offering a visual reference for the experimental procedures and evaluation metrics used in this study.

FIGURE 3.

This study presents a comparative analysis of YOLOv8, YOLOv9, and YOLOv10 models used for detecting defects in powder beds. A comprehensive workflow is detailed, encompassing data collection, model training, and validation processes.

A. Defect Detection Model Based on Yolo

In the context of additive manufacturing, time is a critical factor that determines both printing efficiency and cost. Therefore, rapid identification in automated powder bed defect detection is essential. The YOLO algorithm, which completes detection with a single image pass, enables real-time processing and is particularly well-suited for applications with stringent latency requirements. Applying the latest YOLO model to powder bed defect detection not only significantly improves detection efficiency but also has the potential to profoundly impact the automation of the Laser Powder Bed Fusion (LPBF) process, further advancing the development and application of additive manufacturing technologies. The following sections will provide a detailed overview of the latest YOLO models.

Released in January 2023, YOLOv8 introduced significant innovations that enhance the model’s efficiency, accuracy, and lightweight nature [25]. Its backbone network incorporates the concept of CSPNet [26], replacing the C3 module from YOLOv5 with a more lightweight C2f module while retaining the Spatial Pyramid Pooling Fast (SPPF) module to improve feature extraction efficiency. The Path Aggregation Network (PAN) has been optimized by removing the convolutional structure from the upsampling stage, making the network architecture more streamlined and efficient. Additionally, YOLOv8 features a Decoupled-Head design, allowing for the independent optimization of classification and regression tasks, thereby enhancing overall performance. The model employs an Anchor-Free design, simplifying the training process and increasing flexibility in handling objects of different scales. In terms of the loss function, YOLOv8 combines Distribution Focal Loss (DFL) [27] and CIoU [28] to optimize localization loss, improving the ability to handle class imbalance and localization accuracy. Finally, YOLOv8 introduces the Task-Aligned Assigner matching method, replacing the traditional Intersection over Union (IoU) and unilateral allocation approaches. This adaptation better accommodates complex target distributions and improves the accuracy and efficiency of sample matching. These innovations significantly enhance the accuracy and robustness of object detection while maintaining efficient computation and real-time performance. The YOLOv8 network architecture is illustrated in Figure 4.

FIGURE 4.

The diagram of the YOLOv8 network architecture.

YOLOv9 [29] introduces significant innovations in the field of object detection through the incorporation of Programmable Gradient Information (PGI) and the Generalized Efficient Layer Aggregation Network (GELAN). PGI addresses the issue of information loss in deep networks by ensuring that the network retains complete input information during the learning process. This approach provides more reliable gradient information and improves the accuracy of weight updates, significantly enhancing object detection accuracy and making real-time, high-precision object detection feasible. Simultaneously, GELAN optimizes the network structure through gradient path planning, utilizing traditional convolution operators to achieve higher parameter utilization efficiency than existing state-of-the-art methods. This design not only improves model performance but also ensures model efficiency, enabling YOLOv9 to achieve unprecedented accuracy and speed while maintaining a lightweight structure. The YOLOv9 network architecture is illustrated in Figure 5.

FIGURE 5.

The diagram of the YOLOv9 network architecture.

As the latest model in the YOLO series, YOLOv10 was released on May 23, 2024 [30], introducing multiple innovations that significantly enhance the performance and efficiency of real-time object detection. First, YOLOv10 incorporates a consistent dual-assignment strategy, combining one-to-many and one-to-one annotation methods. This approach eliminates the dependency on Non-Maximum Suppression (NMS), thereby reducing inference latency [31]. Secondly, YOLOv10 employs a comprehensive efficiency-accuracy driven design, including lightweight classification heads, spatial-channel decoupled down sampling, and a rank-guided block design to reduce computational overhead and minimize information loss. Additionally, the model introduces large-kernel convolutions and partial self-attention (PSA) modules, which expand the receptive field and improve global representation learning capabilities. These enhancements enable YOLOv10 to demonstrate outstanding performance on standard datasets like COCO. For instance, YOLOv10-s achieves a speed 1.8 times faster than RT-DETR-R18 at similar Average Precision (AP), with parameters and FLOPs reduced by 2.8 times. YOLOv10-b offers 46% lower latency and 25% fewer parameters compared to YOLOv9-C. Overall, YOLOv10 maintains high precision while achieving greater parameter utilization efficiency and lower inference latency, making it suitable for a variety of applications requiring rapid response. The YOLOv10 network architecture is illustrated in Figure 6.

FIGURE 6.

The diagram of the YOLOv10 network architecture.

Overall, YOLOv8, being the earliest released model, has accumulated extensive user feedback and community support, which may result in strong reliability and performance in powder bed defect detection. YOLOv9, with the introduction of PGI and the GELAN, offers advantages in defect recognition accuracy. In contrast, YOLOv10 excels in post-processing speed and model parameter efficiency by eliminating the dependency on NMS. The YOLOv8, YOLOv9, and YOLOv10 models used in this study are available on GitHub at the following addresses: YOLOv8: https://github.com/ultralytics/ultralytics, YOLOv9: https:// github.com/WongKinYiu/yolov9, and YOLOv10: https:// github.com/THU-MIG/yolov10.

B. Dataset Preparation

In the dataset preparation phase, two high-resolution cameras were employed to capture images of the powder bed in an SLM250 machine. The cameras used were VISION3D PowerScan-5M330 models, and the substrate was a square with dimensions of 250 mm $\times 250$ mm. The imaging setup included two monochrome cameras, each with a resolution of $2448\times 2048$ pixels, positioned at different angles to capture multi-angle views of the powder bed defects. Each image was captured as a single channel ($2448\times 2048\times 1$ ). The experimental setup is illustrated in Figure 7.A total of 2,477 powder bed images were collected, with four defect types artificially simulated based on real defects observed in previous studies of Selective Laser Melting (SLM) processes [8], [10] and those commonly encountered during real-world printing. This approach replicates the typical defects encountered in actual printing processes. Although these defects are not frequently observed in real printing, their occurrence can have a significant impact on part quality. Therefore, it is essential to evaluate the performance of various real-time detection algorithms, which could trigger corrective actions such as powder bed re-coating or pausing the printing process upon defect detection. The defects were categorized into four types: Ditches (420 images), Stacking (600 images), Insufficient Spreading (800 images), and Craters (657 images). Preparing a comprehensive dataset is one of the most crucial steps in deep learning [32], [33]. The core of this study is defect detection using the YOLO model, rather than pixel-wise segmentation. Compared to free-form annotations, drawing bounding boxes is a less complex task for small-sized defects. Pixel-wise annotations may not provide a significant advantage for segmenting small defects [20], and since the goal is to detect defects for powder re-coating, the precision required for defect localization does not need to be at the pixel level, as long as the detection accuracy is ensured. The specific defects are shown in Figure 8.

FIGURE 7.

Experimental setup for powder bed data collection.

FIGURE 8.

Illustrations of various defect samples: (a) and (b) represent Ditches, (c) and (d) represent Insufficient Spreading, (e) and (f) represent Craters, and (g) and (h) represent Stacking. Each defect type is captured by cameras from two different angles. The red dashed boxes indicate the defect locations.

C. Dataset Annotation

The second step in dataset preparation is annotation, which involves labeling the images so that the deep learning network can identify the regions of interest. In this study, surface defects in the powder bed were annotated using the bounding box method. The images were annotated with the Make Sense online annotation tool, where a rectangular box was drawn around each defect area on the powder bed surface. The annotated files were saved in XML format, which facilitates subsequent data augmentation. Specifically, in the dataset, yellow rectangles indicate ditch areas, typically caused by damage to the recoating blade or particles dragging across the powder bed. The red rectangles highlight regions of insufficient powder distribution, caused by the recoating blade’s inability to carry enough powder, leading to inadequate coverage on the substrate. This incomplete spreading can create disturbances that progressively deepen into the powder layer [34]. Green rectangles mark crater areas, which result from powder pits around the part, where vibrations during recoating compact the surrounding powder. Blue rectangles indicate stacking areas, where uneven heating during powder spreading leads to irregular powder accumulation defects. The annotated defects are shown in Figure 9.

FIGURE 9.

Examples of powder bed defect annotations include: (a) Ditches, (b) Insufficient Spread, (c) Craters, (d) Stacking.

D. Dataset Augmentation

The third step in dataset preparation involves data augmentation, which expands the original dataset through a series of transformations to generate new training samples. Data augmentation is a common method to improve model accuracy, especially when data is limited, as it increases the number of image samples and enhances the diversity of defects, thereby improving the robustness of deep learning networks [35]. Common data augmentation techniques include noise addition, flipping, and rotation [36]. This study incorporates seven augmentation modes: rotation, translation, cutout, flipping, cropping, brightness adjustment, and noise enhancement, which can be randomly combined. Below is an overview of these augmentation methods:

Rotation: Images are rotated by a given angle and scale factor using OpenCV to calculate the rotation matrix. Target box coordinates are adjusted to match the rotated image.

Translation: Images and target boxes are randomly translated to ensure all target boxes are included. The minimum bounding box is calculated, and a translation distance is generated. An affine transformation is used to translate the image and update target box coordinates.

Cutout: Random regions of the image are occluded to enhance diversity. The IoU of the occluded region and the target box is calculated to ensure the overlap is less than a threshold, and the occluded region is set to black.

Flipping: Images and target boxes are randomly flipped. A random value determines the flip method (horizontal, vertical, or diagonal), and target box coordinates are updated accordingly.

Cropping: Images are cropped to include all target boxes. The minimum bounding box is calculated and randomly expanded, preventing the cropped area from being too small or exceeding the image boundary. Target box coordinates are adjusted to fit the new size.

Brightness Adjustment: Brightness is adjusted to enhance diversity. A random number between 0.35 and 1 is generated, and a blank image is created to adjust brightness using a weighting function.

Noise Enhancement: Gaussian noise is added to the images to increase diversity, using the current time as a random seed. The pixel values of the output range between [0,1].

The augmentation techniques, including rotation, translation, cutout, flipping, cropping, brightness adjustment, and noise enhancement, were applied with careful consideration to avoid introducing bias or leading to overfitting. Specifically, for rotation, translation, and cropping, we ensured the preservation of all bounding boxes by adjusting the transformation parameters to prevent the loss of any objects. For the cutout operation, we carefully monitored the Intersection over Union (IoU) between the cutout region and the bounding boxes, discarding any cutout with excessive overlap with the objects. The flipping and brightness adjustment operations were applied to simulate real-world variations, while the noise enhancement was designed to mimic realistic sensor noise, ensuring that it did not distort the defect patterns.

Each image is expanded to five images, employing at least one of the aforementioned augmentation methods at random. The augmented powder bed defect dataset contains a total of 12,385 images, with the test set increasing to 2,479 images. An example of data augmentation is shown in Figure 10, with the overall process flow illustrated in Figure 11. The four types of powder bed defects were manually annotated, and the dataset was subsequently augmented. After augmentation, the data was randomly split for further analysis.

FIGURE 10.

Illustration of data augmentation effects. The blue dashed boxes represent five augmented defect images, each randomly enhanced using at least one data augmentation technique. (a) Original image; (b) image after applying cutout and translation; (c) image after applying cutout, flipping, and rotation; (d) image after noise enhancement; (e) image after applying cutout, translation, and flipping; and (f) image after brightness adjustment and rotation.

FIGURE 11.

The workflow for expanding the powder bed defect dataset using data augmentation methods.

After completing the data augmentation process, the annotated files were converted to TXT format. The dataset was then divided into training, validation, and test sets. In this study, the images and their corresponding annotation files were randomly split into 70% for the training set, 10% for the validation set, and 20% for the test set. The distribution of defect samples across these sets is shown in Table 1. The dataset used in this study is available for download at https://doi.org/10.57760/sciencedb.16373.

TABLE 1 The Number of Defect Samples After Data Augmentation

E. Evaluation Metrics

The initialization details of the experimental platform are shown in Table 2. After completing model training, evaluating the performance of YOLO models is a critical step. This study employs multiple metrics to comprehensively assess the detection capabilities of YOLO models, including Precision (P), Recall (R), Mean Average Precision (mAP), mAP@[0.50:0.95], the number of Parameters, and the Detection time of a single image [37]. These metrics reflect the model’s performance from different perspectives, providing guidance for model improvement and optimization.

TABLE 2 Hyperparameter Settings for YOLOv8, YOLOv9, and YOLOv10 in Powder Bed Defect Detection

Researchers commonly use Precision and Recall to evaluate the results of object detection networks. Precision is the proportion of correctly detected positive samples out of all samples identified as positive. Its calculation formula is Precision = TP / (TP + FP), where TP represents True Positives and FP represents False Positives. A high precision rate indicates that most of the detected positive samples are indeed true positive samples. Recall is the proportion of correctly detected positive samples out of all actual positive samples. Its calculation formula is Recall = TP / (TP + FN), where FN represents False Negatives. A high recall rate indicates that the model can identify more actual positive samples.

mAP is a standard metric for evaluating the overall performance of object detection models. It is obtained by averaging precision across different recall rates. The formula for AP at a single threshold is as follows:

$$\begin{equation*} \mathrm {AP}=\sum \nolimits _{\mathrm {n}} {{\mathrm {(R}}_{\mathrm {n}}-\mathrm {R}_{\mathrm {n-1}})\mathrm {P}_{\mathrm {n}}} \tag {1}\end{equation*}$$ where Rn and Pn are the recall and precision values at the nth point, respectively. The mAP is the average of the AP values across all categories, calculated as follows: $$\begin{equation*} \mathrm {mAP}=\frac {1}{\mathrm {N}}\sum \nolimits _{\mathrm {i=1}}^{\mathrm {N}} {\mathrm {A}\mathrm {P}_{\mathrm {i}}} \tag {2}\end{equation*}$$ where N is the number of categories.

The multi-scale average precision is an extended version of mAP, evaluating the model’s performance at different IoU thresholds ranging from 0.50 to 0.95, with a step of 0.05. The mAP@0.5:0.95 is calculated as follows:

$$\begin{align*} \mathrm {mAP}{\text {@}}0.5:0.95=\frac {{\mathrm {mAP}}_{0.50}+{\mathrm {mAP}}_{0.55} {\ldots +} {\mathrm {mAP}}_{0.95}}{\mathrm {N}} \tag {3}\end{align*}$$

The F1 score combines Precision and Recall, providing a comprehensive performance measure for the model [38]. It is calculated using the formula:

$$\begin{equation*} \mathrm {F1}=\frac {2\times \mathrm {Precision\times Recall}}{\mathrm {Precision+Recall}} \tag {4}\end{equation*}$$

The performance metrics P, R, mAP, mAP@0.5:0.95, and F1 score are key indicators of model efficacy, with higher values representing superior model performance. “Params” refers to the total number of trainable parameters within a model. The number of parameters directly impacts the model’s storage requirements and computational complexity. A smaller number of parameters is advantageous for deploying models on resource-constrained devices [39]. Additionally, the evaluation involves analyzing the preprocessing, inference, and post-processing speeds of each model configuration, as these metrics are critical for real-time object detection systems. Preprocessing speed determines how quickly a model prepares an image for detection, inference speed measures the time taken to identify objects within the image, and post-processing speed reflects the time required for the model to finalize the output after detection [40]. The total time for processing an image is the sum of these three stages. Each phase within these stages is critical for the effective operation of rapid prototyping manufacturing applications, where timely and accurate detection significantly impacts decision-making and resource management. The computational efficiency of these models directly affects their practical application in powder bed detection systems, making a thorough evaluation of computational efficiency essential.

F. Training Settings

In this work, our computer system runs on Windows 10 (Microsoft Corporation, Washington, United States) with an Intel Core i9-10980XE 3.00 GHz CPU (Intel Corporation, California, United States) and an NVIDIA RTX A6000 GPU with 47.5 GB of memory (NVIDIA Corporation, California, United States). The implementation was carried out in Python version 3.8 (Python Software Foundation) using the open-source PyTorch framework (Facebook Corporation, California, United States). The environment is configured with CUDA 12.1 and PyTorch 2.2.1 to support GPU utilization. To effectively train the 16 object detection models, we used consistent hyperparameters across all experiments. A batch size of 32 was selected to balance computational efficiency and convergence speed, enabling the model to learn effectively without exceeding GPU resource limits. The Stochastic Gradient Descent (SGD) optimizer was chosen for its robustness and efficiency in dealing with sparse gradients, which is particularly beneficial for object detection tasks. All other hyperparameters were set to YOLO’s default values, as this study primarily aims to assess the performance of various YOLO models in powder bed defect detection, rather than focusing on hyperparameter optimization. The specific hyperparameters are detailed in Table 2.

SECTION III. Results and Discussion

A. Performance of Defect Detection Models

In this study, we conducted a comprehensive evaluation of the performance of 16 model configurations on a powder bed defect detection dataset (including Ditches, Stacking, Insufficient Spreading, and Craters) without applying data augmentation. None of the models used pre-trained weights, and testing was performed on a dataset containing 496 images. In Table 3, bolded values indicate the best performance for the corresponding parameter among the three YOLO models. An upward arrow indicates that higher values are better for that metric, while a downward arrow indicates that lower values are preferable. The detailed results are presented in Table 3.

TABLE 3 The Comprehensive Performance of 16 YOLO Model Configurations in Powder Bed Defect Detection

We conducted a comparative analysis of detection accuracy across different YOLO configurations, highlighting subtle performance variations in precision and recall metrics. This study evaluated the performance of various YOLO models in detecting powder bed defects. YOLOv9-s and YOLOv8-l demonstrated the highest precision at 0.928, making them suitable for applications demanding exceptionally high detection accuracy. YOLOv9-m demonstrates the strongest overall performance, achieving the highest recall (0.896), mAP50 (0.947), and F1 score (0.910). The introduction of PGI and the GELAN in YOLOv9 makes it well-suited for applications requiring both high detection rates and precision. YOLOv8-l showed superior performance in mAP50-95 (0.551), indicating its robust detection capabilities across various IoU thresholds. YOLOv8-n had the shortest detection time (12.4ms), and YOLOv9-t featured the smallest parameter size (2.6M). These results offer multiple model choices for powder bed defect detection, allowing for trade-offs between detection accuracy, speed, and resource consumption based on practical requirements.

Figure 12 illustrates the precision and recall of YOLO algorithms across all configurations. In the YOLOv8 series, YOLOv8-l stands out with the highest precision (0.928), indicating its superior effectiveness in recognizing defect targets. YOLOv8-m exhibited the highest recall (0.893), suggesting it is the most effective at identifying all relevant instances, though it may have a higher false positive rate. In the YOLOv9 series, YOLOv9-s achieved the highest precision (0.926) but had a lower recall (0.860), while YOLOv9-m demonstrated better overall performance with a precision of 0.924 and recall of 0.896, making it the best-performing configuration among the five YOLOv9 tests. For YOLOv10, the YOLOv10-l configuration achieved the highest precision (0.926), indicating its accuracy in correctly identifying objects without excessive false detections. However, it showed a relatively lower recall (0.834), suggesting that some true positives might be overlooked.

FIGURE 12.

The scatter plot illustrates the precision and recall of all configurations of the YOLOv8, YOLOv9, and YOLOv10 object detection algorithms in powder bed defect detection.

Figure 13 presents the mAP@0.50 evaluation results for different YOLO model configurations. Among all evaluated configurations, the YOLOv9-m model achieved the highest score, reaching 0.947. Notably, several YOLOv8 configurations also performed exceptionally well in terms of mAP@0.50, with YOLOv8-l achieving a score of 0.946, demonstrating YOLOv8’s high accuracy in object detection tasks. Within the YOLOv10 series, YOLOv10-m attained the highest score with an mAP@0.50 of 0.918, followed closely by YOLOv10-b and YOLOv10-s, with scores of 0.916 and 0.915, respectively. Although these configurations showed strong performance, their scores were still below the peak of YOLOv9, indicating that specific enhancements in YOLOv9 may have significantly boosted its accuracy.

FIGURE 13.

The bar chart displays the mAP@50 scores for all 16 configurations of YOLOv8, YOLOv9, and YOLOv10 used in powder bed defect detection.

In comparison, the performance range of YOLOv8 configurations is broader. YOLOv8-l achieved the highest mAP@0.50 in its group at 0.946, while the YOLOv8-x configuration scored lower at 0.938. It is noteworthy that the YOLOv8-x model has the largest number of parameters and floating-point operations, with 68.2M and 257.8B, respectively, making it the most computationally intensive model among the 16 models in our study. However, despite its higher computational complexity, it did not result in a significant improvement in accuracy. This may be due to the limited dataset size or the relatively simple characteristics of powder bed defects, where increased model complexity does not yield additional benefits. Therefore, we plan to retrain the network using a dataset augmented with enhanced data in future research.

In the evaluation of computational efficiency, particularly in image preprocessing speed, YOLOv9 emerged as the optimal configuration, achieving the fastest preprocessing speed of 0.3 milliseconds. Theoretically, the preprocessing speed of different models should be consistent, with YOLOv8 and YOLOv9 expected to have comparable performance. The variations in preprocessing speed observed might be due to random error rather than fundamental differences in the model architectures.

In line with the discussion on preprocessing speed, YOLOv8 configurations also excelled in inference speed analysis. Notably, YOLOv8-n achieved an exceptionally fast inference speed of 7.5 milliseconds, outperforming the fastest configurations in the YOLOv9 and YOLOv10 series. Specifically, the fastest YOLOv9 configuration, YOLOv9-t, had an inference speed of 23.6 milliseconds, while the fastest YOLOv10 configuration, YOLOv10-n, had an inference time of 11.8 milliseconds. These results highlight YOLOv8-n’s significant performance advantage in inference capabilities. This exceptional computational speed during the critical inference stage underscores YOLOv8-n’s robustness, indicating that earlier versions of the YOLO series, particularly YOLOv8, still hold a distinct advantage in applications requiring rapid and accurate object detection. These insights are crucial for optimizing real-time applications where fast decision-making is essential, reaffirming YOLOv8’s relevance in current technology.

Table 4 systematically lists the detailed performance metrics for all configurations of YOLOv8, YOLOv9, and YOLOv10, providing a comprehensive overview of the computational efficiency of each configuration. YOLOv10 excelled in post-processing speed, setting a new benchmark for rapid image processing completion. Specifically, all YOLOv10 configurations, except for YOLOv10-x, had post-processing times of just 0.7 milliseconds, significantly outperforming the fastest configurations in YOLOv9 and YOLOv8, demonstrating its advanced capability in efficiently handling the final stages of object detection. Overall, while YOLOv8 configurations surpassed YOLOv9 and YOLOv10 in inference speed, YOLOv10 emerged as the leader in post-processing speed. This is due to YOLOv10’s ability to achieve highly advanced performance by eliminating NMS and optimizing various model components, significantly reducing computational overhead [30].

TABLE 4 The Computational Speed of YOLOv8, YOLOv9, and YOLOv10 Configurations for Powder Bed Defect Detection

The comparison of processing speeds among different models is illustrated in Figure 14, where YOLOv9 performs best in preprocessing speed, YOLOv8 excels in inference speed, and YOLOv10 leads in post-processing speed. Thus, selecting the appropriate model should be based on the specific requirements of the application scenario.

FIGURE 14.

The line chart compares the preprocessing speed, inference speed, and post-processing speed of the YOLOv8, YOLOv9, and YOLOv10 configurations.

B. Influence of Training Data on the Yolo Detection Accuracy

To verify whether an increase in data volume can enhance model performance, we applied augmentation to the powder bed dataset. Specifically, we employed a random combination of seven augmentation techniques: rotation, translation, cutout, flipping, cropping, brightness adjustment, and noise enhancement. The test set was expanded to 2,479 images. The results on the test set after training are shown in Table 5. These results provide important insights into how data augmentation affects YOLO performance and offer valuable guidance for our future research. We look forward to further exploring these findings in future work and applying this knowledge in practice to achieve more efficient and accurate powder bed defect detection. Among the YOLOV8, YOLOV9, and YOLOV10 series models, the optimal models are YOLOV8-x, YOLOV9-e, and YOLOV10-m, respectively. YOLOV8-x demonstrates the best performance in terms of detection accuracy and speed, making it suitable for applications requiring high speed and precision. YOLOV9-e excels in detection accuracy, with the highest recall rate and mAP@50, but is slower, making it ideal for scenarios where accuracy is paramount and time constraints are less critical. YOLOV10-m offers a balanced performance in both accuracy and speed, with the added advantage of having the smallest parameter size, making it versatile for various applications.

TABLE 5 The Performance of 16 YOLO Model Configurations in Powder Bed Defect Detection After Data Augmentation

From the table results, we can clearly observe the significant improvement in network performance due to data augmentation. Compared to the performance before data augmentation, the results of the improvement rate are shown in Figure 15. In the Figure 15, all metrics are designed so that higher values indicate greater performance improvement after data augmentation. Specifically, precision improved by 9.4% for the YOLOv10-n version, and recall improved by 12% for the YOLOv10-x version, indicating a substantial enhancement of the YOLOv10 network due to data augmentation. For mAP@50, YOLOv10-x improved by 6.5%, while mAP@50-95 saw a 40% increase for YOLOv8-x. On average, YOLOv10 exhibited the most significant improvement in mAP. The F1 score for YOLOv10-b increased by 9%. Compared to YOLOv8 and YOLOv9, YOLOv10 has been extensively optimized, particularly in the PSA and large kernel convolutions, significantly enhancing its global modeling capabilities and feature extraction efficiency. These design improvements allow YOLOv10 to better adapt to diverse feature patterns in data-augmented applications, achieving an optimal balance between performance and efficiency. Consequently, YOLOv10 exhibits substantial performance gains in powder bed defect detection tasks. In terms of the detection time for a single image, there was little change for YOLOv10 and YOLOv9, but the detection time decreased to varying degrees for all YOLOv8 models, with YOLOv8-l experiencing a 28% reduction. The reduction in detection time primarily occurred in the post-processing stage. This may be attributed to YOLOv8 undergoing multiple updates and optimizations, resulting in architectural enhancements, effective responsiveness to data augmentation, and good hardware compatibility.

FIGURE 15.

The improvement rates in the performance of 16 YOLO model configurations for powder bed defect detection after data augmentation are as follows: (a) improvement rates in precision and recall, (b) enhancement rates in mAP@50 and mAP@50-95, (c) increase rate in the F1 score, and (d) reduction rate in the time required to detect a single image.

As demonstrated in the confusion matrices shown in Figure 16, the analysis indicates that YOLOv9-e emerges as the most optimal model for powder bed defect detection. It performs exceptionally well in detecting Craters and Stacking defects, while maintaining a favorable balance between high accuracy and relatively low false positive rates across all defect categories. YOLOv8-x, though a strong contender, exhibits excellent performance in detecting Ditches and Stacking defects but faces challenges in accurately identifying Craters. On the other hand, YOLOv10-m, while delivering consistent results, shows slightly reduced accuracy in Craters detection and exhibits higher false positive rates, rendering it less optimal when compared to YOLOv8-x and YOLOv9-e.

FIGURE 16.

Confusion matrices of the YOLO models on the test set after data augmentation. (a) YOLOV8X, (b) YOLOV9E, and (c) YOLOV10M.

During the actual testing phase, we selected the models with the highest F1 scores from each configuration for further evaluation. Four images, each representing a different defect category from the test set, were randomly selected. These images were tested using both the pre- and post-data augmentation weights. The results are shown in Figure 17 and 18. A comparison of the results reveals that models using data augmentation exhibited higher confidence levels in detecting various defect types, with more accurate and stable detection outcomes. Data augmentation enhanced the diversity of the training data, which improved the models’ generalization capabilities. This, in turn, led to better robustness and precision in real-world detection scenarios. In contrast, models that did not utilize data augmentation performed relatively poorly in some cases, with a higher likelihood of missing defects or producing low-confidence detections.

FIGURE 17.

Visualization results of YOLOv8-x, YOLOv9-e, and YOLOv10-m models on four types of powder bed defects without data augmentation.

FIGURE 18.

Visualization results of YOLOv8-x, YOLOv9-e, and YOLOv10-m models with data augmentation on four types of powder bed defects.

Figure 19 presents a comparison of Grad-CAM(Gradient-weighted Class Activation Mapping) images generated by the YOLOv8-x model, using gradients from the eighth layer, before and after data augmentation for four types of defects. The top row shows Grad-CAM images before data augmentation, while the bottom row shows Grad-CAM images after data augmentation. In each image, the red region represents the parts of the image that the model is most focused on, while the blue region indicates areas of lesser focus. By comparing the two, it can be observed that after data augmentation, the model’s localization of defects becomes more accurate, with red regions becoming more concentrated at the defect locations. This indicates that data augmentation significantly enhances the model’s sensitivity to defect areas, leading to improved detection accuracy. This improvement highlights the positive impact of data augmentation in enhancing the model’s ability to recognize small defects, especially in noisy or complex backgrounds.

FIGURE 19.

Grad-CAM image comparison analysis. The top row shows Grad-CAM images before data augmentation, while the bottom row shows Grad-CAM images after data augmentation.

This study focused on four significant types of powder bed defects, as these defects often have a considerable impact on the quality of the final product. However, in future work, we aim to construct a powder bed dataset encompassing a broader range of defect types, including those that occur naturally during the printing process. This will enhance our proposed method’s detection capability across a more diverse array of defects. Such an expansion will help us gain a more comprehensive understanding of the powder bed defect detection problem and provide more effective and accurate solutions. We look forward to further exploring and validating these observations in future research and applying this knowledge in practice. Overall, our results demonstrate that a combination of effective data augmentation strategies and appropriate model selection can significantly improve the performance and efficiency of deep learning models in powder bed defect detection tasks. We anticipate further exploration of these findings in future work to achieve more efficient and accurate powder bed defect detection.

Currently, the trained weight files have been converted to the ONNX format and are being utilized by the C# code for real-time defect detection on our industrial hardware, which is configured with an Intel Core i7-9700 3.00 GHz CPU (Intel Corporation, California, United States), an Intel UHD Graphics 630, 8 GB of memory, and 16 GB of RAM. Upon detecting a defect, the system automatically reapplies a fresh layer of powder to the affected area. If the defect persists after three consecutive re-coating attempts, the system will halt the printing process and trigger an alert to notify the operator. This approach is designed to minimize the impact of defects on the overall quality of the printed object, ensuring that any issues are promptly addressed before they can affect subsequent layers. This integration demonstrates the efficiency and effectiveness of the defect detection process, ensuring its smooth operation within a live industrial environment, with real-time detection time consistently under 100 ms.

In this study, we conducted a comprehensive evaluation of various configurations of three state-of-the-art YOLO object detection models (YOLOv8, YOLOv9, and YOLOv10) to assess their effectiveness in detecting defects in the LPBF process. While the proposed approach demonstrated strong detection performance, there are several limitations that warrant further improvement: (1) Data Augmentation: Future work should involve using defects generated from actual printing processes, followed by the application of more advanced data augmentation techniques, such as generating diverse samples using GANs (Generative Adversarial Networks) [41]. (2) Detection Efficiency: Smaller and more efficient detection models are often required to meet the demands of real-time monitoring. Given that LPBF powder bed defect detection is a relatively straightforward task, developing more compact models can improve training efficiency and inference speed, facilitating the deployment of deep learning methods in LPBF monitoring systems. Our research does not focus on improving the YOLO model; instead, it concentrates on evaluating the most optimal models for powder bed defect detection. Model improvement will be the focus of my future work. (3) Semi-Supervised and Unsupervised Learning: Reducing the time and resources required for manually labeling training samples could be achieved by employing unsupervised or semi-supervised learning methods. For instance, GAN-based defect detection [42] or semi-supervised approaches could be used to detect anomalies in LPBF-produced parts [43].

SECTION IV. Conclusion

This research makes significant contributions to the field of powder bed defect detection in the LPBF process, with the following key innovations:

1. Specialized Dataset Construction: We developed a dataset specifically for powder bed defect detection, covering four common defects: Ditches (420 images), Insufficient Spreading (800 images), Craters (657 images), and Stacking (600 images). Each defect type was carefully annotated, providing a rich set of samples for training and evaluating deep learning models. This dataset addresses a gap in the availability of public datasets in this domain.

2. Multi-Model Evaluation: We conducted a comprehensive evaluation of 16 advanced deep learning models on this dataset, exploring their performance in defect detection tasks. By comparing these models, we highlighted differences in accuracy, speed, and overall performance, offering valuable insights for future research and practical applications.

3. Data Augmentation Experiments: We performed data augmentation on the original dataset, expanding each defect type fivefold through techniques such as rotation, scaling, and noise addition, resulting in a total of 12,385 images. The experimental results demonstrated that data augmentation significantly improved detection accuracy and model performance, particularly in scenarios with limited defect samples, validating the effectiveness of data augmentation techniques in enhancing model capabilities.

4. Model Performance Analysis: We conducted an in-depth analysis of the performance of YOLO series models in defect detection. Notably, the YOLOv9 model achieved the highest mAP@0.50 score, showcasing exceptional detection accuracy. YOLOv9-e particularly excelled in Precision and Recall metrics, with scores of 0.965 and 0.975, respectively. Additionally, YOLOv10 exhibited outstanding post-processing speed, making it well-suited for data-rich environments; YOLOv8-n excelled in inference speed, and YOLOv9-t led in preprocessing speed. These analyses provide concrete guidance for practical deployment and application.

In this study, powder bed defects such as Ditches, Stacking, Insufficient Spreading, and Craters are identified based on visible anomalies. At this stage, these defects are treated independently of the printed part’s location, and future work will address the correlation between defects and part positioning. Through this study, we not only provide a high-quality powder bed defect detection dataset but also validate the potential of the latest deep learning models in this field. Moreover, we have demonstrated the effectiveness of data augmentation techniques in enhancing model performance. These innovations contribute to improving the manufacturing quality and process stability of the LPBF technique, thereby advancing the development of additive manufacturing technologies.

Data Availability

The data that support the findings of this article are available from the corresponding author upon request.

Conflict of Interest

The authors declare that they have no conflict of interest.