A Tightness Detection Method for Railway Fasteners Based on RGB-P Bimodal Data

Rail fasteners are critical components in railways and are inspected periodically to ensure train safety. Visual inspections using machine technology are commonly employed to identify any abnormal fasteners in rail tracks. However, it can be challenging to determine the tightness of fasteners based on unimodal data. This is where multimodal data fusion technology comes into play, offering a viable solution. In this article, a novel method based on red, green, blue (RGB)-P bimodal data fusion is proposed to detect the tightness of fasteners. First, a 3-D laser sensor is employed to collect the profiles of the railway, and bimodal data which consists of RGB depth images and point clouds (RGB-P for short) is constructed based on the profiles. Then, the clip of the fastener is rapidly segmented and the clip’s skeleton is extracted from the RGB depth image. The 2-D skeleton is fused with the point cloud to quickly obtain the 3-D skeleton of the clip. Finally, the features of the 3-D clip skeleton are extracted to calculate the clip gap, and the tightness of the fastener is detected based on the clip gap. Experiments are conducted on ballast and ballast-less railways, including WJ-7, WJ-8, and WJ-2 types of fasteners, to evaluate the performance, robustness, and precision of the fastener tightness detection. The experimental results demonstrate that our proposed method can accurately measure the clip gap with the measurement error and root mean square error (RMSE) both less than 0.1 mm, and is robustness to the railway environment. When the measurement error is within ±0.1 mm, the overall detection rate (DR) of fastener tightness detection exceeds 99.4%, and the detection speed is not less than 10.05 m/s. The detection system can be installed on rail inspection vehicles to detect the tightness of fasteners in real-time and improve efficiency of the railway maintenance.