Learning Pothole Detection in Virtual Environment

Pothole detection is an important function in autonomous vehicles, which can help vehicles to avoid dangerous traps on roads, or change suspension to make passengers more comfortable. However, it is challenging to train a high quality pothole detector, mainly due to the difficulty of collecting training data. Sending automobiles with cameras to record videos of potholes is time consuming, expensive, and may lead to unexpected accidents. To address this issue, we leverage the recent emerging virtualto-real learning and use latest virtual reality technology to train a pothole detector. We develop a pothole generation system that can generate holes with various shapes, sizes, and depths. The virtual pothole images are added to the training dataset, and the detector performance is evaluated on real data. Experiment results show that virtual pothole images can successfully increase the overall detection accuracy and enable users to train detectors with less real data.