Automatic Map Update Using Dashcam Videos

Autonomous driving requires 3-D maps that provide accurate and up-to-date information about semantic landmarks. Since cameras present wider availability and lower cost compared with laser scanners, vision-based mapping solutions, especially, the ones using crowdsourced visual data, have attracted much attention from academia and industry. However, previous works have mainly focused on creating 3-D point clouds, leaving automatic change detection as open issue. We propose a pipeline for initiating and updating 3-D maps with dashcam videos, with a focus on automatic change detection based on comparison of metadata (e.g., the types and locations of traffic signs). To improve the performance of metadata generation, which depends on the accuracy of 3-D object detection and localization, we introduce a novel deep learning-based pixelwise 3-D localization algorithm. The algorithm, trained directly with Structure from Motion (SfM) point cloud data, accurately locates objects in 3-D space by estimating not only depth from monocular images but also lateral and height distances. In addition, we also propose a point clustering and thresholding algorithm to improve the robustness of the system to errors. We have performed experiments with different types of cameras, lighting, and weather conditions. The changes were detected with an average accuracy above 90%. The errors in the campus area were mainly due to traffic signs seen from a far distance to the vehicle and intended for pedestrians and cyclists only. We also conducted cause analysis of the detection and localization errors to measure the impact from the performance of the background technology in use.