A disocclusion filling method using multiple sprites with depth for virtual view synthesis

Depth image based rendering (DIBR) is an important technique to generate virtual view images with limited 3-D data. However, disocclusion is a critical problem that the regions occluded by foreground objects become visible in virtual views, which is difficult to be visual-plausibly inferred. In this paper, we propose a novel temporally consistent filling method using multiple sprites with depth (MSD) to fill the disocclusions faithfully. MSD stores the background as well as intermediate foreground objects of past frames in multiple sprites, so that amounts of pixels in disocclusions can be recovered from MSD. Moreover, we also introduce a method to recognize the source space for exemplar based inpainting to fill the remaining disocclusions. The experimental results show the proposed method achieves objective and subjective improvement compared with the state-of-the-art methods. The synthesized sequences by the proposed method have higher spatio-temporal consistency.