Improving Depth Estimation Using Map-Based Depth Priors

Accurate scene depth is fundamental for robot scene understanding as it adds spatial reasoning. However, accurate scene depth often comes at the cost of expensive additional depth sensors. In this letter, we propose to use map-based depth data as an additional input instead of expensive depth sensors. Such an approach is especially appealing in autonomous driving since map-based depth is commonly available from high-definition maps. To validate this approach, we propose a mapping method that works with common autonomous driving datasets and allows for precise localization using a mix of GNSS-INS and image-based techniques. Furthermore, we propose an entirely learnable three-stage network that handles foreground-background mismatches between the map-based prior depth and the actual scene. Finally, we validate the performance of our method in comparison to several baseline methods and SOTA depth completion methods receiving map-based depth as an input. Our method significantly outperforms these methods both in quantitative and qualitative results. Moreover, our method achieves better metric-scale predictions compared to image-only approaches.