Multiscale two-view stereo using convolutional neural networks for unrectified images

Two-view stereo problem is a well researched problem in 3D computer vision. Algorithms proposed in the past have focused on rectified stereo images where the epipolar lines are parallel to the horizontal axis. The general problem of computing stereo correspondences for unrectified images without any knowledge of calibration parameters is an important problem but unexplored as yet. Our idea in this paper is to predict depth maps from two unrectified stereo images using a modified Flownet architecture. Since, datasets for depth map reconstruction for unrectified stereo images for deep learning do not exist, we have created a dataset of turn table sequences of 3D models from Google 3D warehouse. Following the concepts of Attention modelling, we implement an architecture for combining correlations computed at multiple resolutions using a simple element-wise multiplication of the correlations to aid the architecture to resolve correspondences for textureless and repeated textured surfaces. Our experiments show both qualitaitve and quantitative improvements of depth maps over the original Flownet architecture.