The Two-Step Method of Pavement Pothole and Raveling Detection and Segmentation Based on Deep Learning

Automatic detection and segmentation of potholes and raveling are crucial for preventive maintenance and ensuring roads structural health. However, the extraction of pavement potholes was mainly based on traditional image processing methods, which have proven to be ineffective and inaccurate. Additionally, the absence of a unified pothole and raveling dataset has resulted in the lack of benchmarks for evaluating various methods. This work makes three contributions to address these problems. Firstly, we have curated the pavement pothole and raveling distress detection and segmentation datasets. Secondly, a two-step pavement pothole and raveling detection and segmentation method was proposed. In the initial step, an automated pavement pothole and raveling detection model was developed using the modified YOLOX. Subsequently, the segmentation model, named dual self-attention segmentation network (DSASNet), was proposed to segment distress by extracting mode-sensitive features from intensity and range images using two parallel Twins-SVT self-attention branches. Moreover, we design a mid-fusion module to adaptively fuse mode-specific and scale-specific features. Finally, a pyramid pooling module (PPM) is connected to further enhance the segmentation capability for potholes and raveling of various sizes and shapes. The F1-score and Intersection over union (IoU) of the proposed DSASNet on the test set are 93.65% and 0.881, respectively, outperforming other baseline methods. Furthermore, we conduct an experiment to quantitatively compare the two-step method with the one-step method using only a single semantic segmentation model. The results demonstrated clear advantages of the proposed two-step method in terms of accuracy and efficiency for pavement pothole and raveling segmentation.