Pothole Detection for Visually Impaired Assistance

The global number of visually impaired is growing fast due to aging world population. People suffering of severe visual impairments face many difficulties in their daily routine. Pavement holes are a major problem for their navigation and walking. The proposed algorithm detects holes in the sidewalks and roads using a Convolutional Neural Network. Starting from the previously published research, this paper proposes a practical solution for pothole detection used in the Navigation Module of the Sound of Vision Lite (SoV Lite) project. YoloV5s, Mobilenet V1 and Mobilenet V2 Lite were trained on Nvidia RTX using the obtained dataset, then deployed for testing on Nvidia Jetson NX. Due to the mobile platform constraints Mobilenet V1 was chosen to be integrated in SoV Lite. Because objects which are closer have higher detection confidence but also because the visually impaired person wearing the system is interested in the dangers close to him, in practice, we limit the detections of negative obstacles within a range of 8m. By creating a region of interest the run time is also enhanced. For training and experiments we used in-house acquired frames using a ZED 2 Stereo Camera as well as publicly available data and annotated it for the specific task of detecting potholes and drains in the pavement and streets. The obtained dataset was augmented and made publicly available.