Direct Optical-Flow-Aware Computational Framework for 3D Reconstruction

In this paper, a direct computational method is presented which combines optical flow and structure from motion (SfM) by putting the SfM problem in the framework of optical flow estimation. In other word, the optical flow is reparametrized in term of the camera's motion and scene's depth, resulting in a similar variation optimization as in optical flow estimation. Meanwhile, three techniques are proposed to improve the accuracy and robustness of the direct approach, including the fast guided interpolation (FGI), the left-right consistency constraint and the soft segment constraint. Experimental results on the Middlebury dataset and KITTI2012 dataset show that the proposed approach can achieve highly-accurate 3D reconstruction with the dense and smooth surface which results in a state-of-the-art performance in optical flow.