Panoramic Stereo Matching Network Based on Bi-Projection Fusion

Stereo Matching from panoramic images is becoming a hot research topic because panoramas have an entire field of view and provide a complete semantic scene. However, due to the nonlinear epipolar constraint, estimating depth from an equirectangular projected panoramic stereo is suboptimal. The cubemap projection is another representation, which is undistorted and has a more straightforward epipolar constraint but introduces discontinuities at the boundaries of the cube. Considering that the two projections are complementary, we proposed an end-to-end trainable network based on bi- projection fusion for stereo depth estimation using 360° images. To resist the interference of the distortion in the equirectangular projection, we constructed cost volumes utilizing epipolar constraints in equirectangular and cubemap projections, respectively. In addition, we designed two fusion modules for the fusion scheme, one of which has high computational accuracy and the other is faster. Numerous experiments and ablation studies are given to validate our approach against the current algorithm.