Rapid Detection of Visual and Structural Defects of Railway Fasteners via RGB-P Bimodal Data Fusion

Railway fasteners ensure the safety of rail operations, but existing methods of the fastener defect detection struggle to identify both visual and structural defects. This paper proposes an effective method based on the RGB depth image and Point cloud (RGB-P) bimodal fusion to enable simultaneous identification of visual and structural defects. A three-dimensional (3D) line laser sensor captures track profiles to generate RGB-P data. A bidirectional spatial map is established for the seamless integration of texture from RGB-depth images and geometric features from point clouds. For visual defects, the You Only Look Once version 8(Yolov8) network detects anomalies in RGB depth images with 100% precision under varying lighting conditions. For structural defects, the 3D skeleton of Middle Bend of Clip is extracted via cross-modal guidance fusion framework, and clip gaps are calculated using dynamic diameter correction via cross-sectional circle fitting to avoid errors from fixed diameter assumptions. Structural defects are detected based on the clip gap. Experiments show that the method achieves a Root Mean Square Error (RMSE) less than 0.1 millimeter (mm) for the clip gap measurement, and structural defect detection precision and recall exceeding 97.5% under ±0.1 mm error tolerance. Our method surpasses existing approaches in precision, recall, and efficiency with an average speed of 41.3 meters per second, which is suitable for the real-time detection of both visual and structural defects of railway fasteners.