PatchMatch-Based Content Completion of Stereo Image Pairs

This paper presents a method for completing target regions ("hole filling") in RGB stereo pairs. It builds upon the state of the art for completing single images by matching to and then blending source patches drawn from the rest of the image. A method is introduced for first completing the respective disparity maps using a coupled partial differential equation based on that of Bertalmio, et al., extended to create mutual disparity consistency. Estimated disparities are then used to guide completion of the missing color image texture. An extension to the coherence-based objective function introduced by Wexler, et al. is then introduced, which not only encourages coherence of the respective images with respect to source images but also stereoscopic consistency between the two. The Patch Match algorithm of Barnes, et al. is extended to cross-image searching and matching. This matching is capable of automatically copying from corresponding unoccluded portions of the other image without requiring an explicit preliminary warping step. Stereoscopic consistency is produced by giving preference to matches with cross-image consistency when blending source patches. Results demonstrate that this method produces better completion than either single-image completion or previous methods for stereo completion.