Hierarchical Semantic Segmentation Based Approach for Road Surface Damages and Markings Detection on Paved Road

Detection of road surface damages, such as potholes, cracks, and markings on a paved road from images captured by a dashcam is an essential issue in developing automatic road inspection systems. When we apply ordinary non-hierarchical semantic segmentation in this task, the system sometimes detects the potholes, cracks, and markings, outside the road area, e.g., in the sky, woods, etc. To address this issue, we propose a method to use a hierarchical structure on semantic segmentation. This method segments an input image in two levels of the layers. Firstly, the first level of the layer classifies the paved road and background. Next, the second level of the layer identifies potholes, cracks, and markings on a paved road area that is identified in the first level of the layer. To obtain a complete segmentation map, we apply the elementwise multiplication to the output of both levels of the layers. The U-Net was used in each semantic segmentation. We compared our method with ordinary non-hierarchical segmentation in terms of F1-score and Intersection over Union (IoU). Results show that our method outperforms the ordinary non-hierarchical segmentation for the overall classes in terms of F1-score and IoU. Compare to the ordinary non-hierarchical segmentation, our method improved the result; (i) from 76% to 85% of F1-score and 61% to 74% of IoU for potholes, (ii) from 62% to 68% of F1-score and 45% to 51% of IoU for cracks, (iii) from 89% to 90% of F1-score and 80% to 82% of IoU for markings.