RSM-YOLOv11: Lightweight Steel Surface Defect Segmentation Algorithm Research Based on YOLOv11 Improvement

Traditional segmentation algorithms fail to effectively handle complex texture backgrounds and diverse defect shapes in steel surface defect segmentation, resulting in insufficient segmentation accuracy (SA), slow speed, and false negatives. To address these issues, this paper proposes an improved YOLOv11 segmentation model, RSM-YOLOv11. The Space-to-Depth Convolution (SPD-Conv) module is introduced to replace the traditional convolutional layer. This module combines the Space-to-Depth (SPD) layer with a Non-strided Convolutional (NSConv) layer, which effectively preserves detailed information in the channels during the downsampling process, avoiding the loss of information associated with traditional convolutional operations and enhancing the segmentation performance for small-sized defects. The Re-parameterizable Block (RepBlock) module is integrated into the backbone and neck networks. During the training phase, the multi-branch structure of this module enriches the feature representation levels and captures steel surface defect (SD) features comprehensively, thereby improving SA. In the inference phase, the multi-branch structure is transformed into a compact single-branch form through reparameterization technology, effectively accelerating the network’s inference speed. Finally, the Multi-scale Channel Attention and Spatial Attention Module (MCASAM) is introduced to replace the original C2PSA module. This module optimizes the channel attention mechanism by replacing the channel attention mechanism in the Convolutional Block Attention Module (CBAM) with the efficient Multi-Scale Convolutional Attention (MSCA). While retaining the spatial localization advantage, it enhances the feature representation capability in the channel dimension, enabling the model to capture multi-scale contextual information and thereby increasing its sensitivity and segmentation ability for steel SD information. Experimental results show that the improved RSM-YOLOv11 model achieves an average SA of 92.3% for three types of defects in the NEU-Seg dataset, which is 6% higher than that of the baseline model. The model size is reduced by 23.81%, and the computational cost is decreased by 15.38%. While maintaining a lightweight structure, it outperforms existing mainstream algorithm models. Additionally, generalization experiments using other types of datasets confirm that the algorithm has good generalization ability.