Road Damage Evaluation via Stereo Camera and Deep Learning Neural Network

Road damage detection and quality evaluation are important processes that keep the public and private road systems running smoothly and accident-free. Currently, most road pavement assessments are done manually, which is slow, costly, and subject to human error. This paper proposes a road quality evaluation system capable of detecting potholes and evaluating the severity. The system contains a stereo camera and a laser diode that is capable of projecting multiple laser dots. The stereo camera collects images while the laser diode projects easily identifiable marks on the pavement. First, the collected images without laser dots are fed into pothole detection neural network (PDNN), which is based on You Only Look Once (YOLO) network, to detect potholes. The detected potholes are outlined with label boxes. Second, the laser dots are projected onto the potholes' surface and the corresponding images with the laser dots are taken by the stereo camera. Third, the laser dots on each picture are identified through image processing and used to calculate the distance from each mark on the potholes' surface to the camera by matching the marks on the left and right images. Finally, a rough 3D model of the pothole is constructed using the distances and the volume of each damaged area are calculated. These steps can be performed at a high speed if the camera and laser control module is built properly. Using only stereo camera and laser dots makes the system inexpensive. Furthermore, our research covers various weather conditions and road pavement materials, including asphalt, concrete, and dirt. The comprehensive experiment results are provided at the end of this paper.