Two-stage Parallax Correction and Multi-stage Cross-view Fusion Network Based Stereo Image Super-Resolution

Stereo image super-resolution (SR) has achieved great progress in recent years. However, the two major problems of the existing methods are that the parallax correction is insufficient and the cross-view information fusion only occurs in the beginning of the network. To address these problems, we propose a two-stage parallax correction and a multi-stage cross-view fusion network for better stereo image SR results. Specially, the two-stage parallax correction module consists of horizontal parallax correction and refined parallax correction. The first stage corrects horizontal parallax by parallax attention. The second stage is based on deformable convolution to refine horizontal parallax and correct vertical parallax simultaneously. Then, multiple cascaded enhanced residual spatial feature transform blocks are developed to fuse cross-view information at multiple stages. Extensive experiments show that our method achieves state-of-the-art performance on the KITTI2012, KITTI2015, Middlebury and Flickr1024 datasets.

I. Introduction

Stereo image super-resolution aims to reconstruct a pair of high-resolution (HR) images from a pair of low-resolution (LR) images, which is crucial to computer vision application and industrial application, such as 3D object detection, security monitoring system and autonomous driving for vehicles. The information from a stereo image pair has been shown to be beneficial in improving super-resolution performance [1]–[5]. However, it is challenging to achieve better stereo image SR performance due to varying parallax and imperfect cross-view information fusion.