Optical-Flow-Aided Self-Supervised Single View Depth Estimation from Monocular Videos

Recent advances in deep-learning-based accurate optical flow (OF) estimation have sparked interest in building hardware dedicated to it, making OF a real-time commodity for downstream computer vision tasks. Relative camera pose, and depth estimations (DE) are closely related to OF, but obtaining one from the others is not a trivial task. While previous works have attempted simultaneous learning of OF and DE, or utilized OF-nets only for camera pose prediction and sparse depth map supervision, we propose OF-aided self-supervised DE, that is, pre-computed OF is used as a network input. One of the main challenges in OF-aided DE is preventing the DE network from learning an incorrect OF magnitude prior, which is valid for rigid regions but breaks for scenes with dynamic objects or static camera sequences. We achieve OF-aided DE by instead transforming the input OF into a 3D surface normals space, which provides a well-informed geometrical input while being invariant to the OF magnitude. In combination with a new training strategy, we show that our models with OF-aided DE achieve state-of-the-art (SOTA) results on the KITTI dataset.