Geometric Warping Error Aware Spatial-Temporal Enhancement for DIBR Oriented View Synthesis

Depth-Image-based Rendering (DIBR) oriented view synthesis is a crucial technique of virtual view synthesis in 3D video. It generates virtual viewpoint images using a few (usually two) reference viewpoints with DIBR based 3D warping. Geometric warping error, often overlooked by existing methods, is inevitably encountered in the 3D warping process due to the integer pixel representationx of images. To reduce the geometric warping error (GWE) in virtual view synthesis, a GWE aware spatial-temporal enhancement framework is proposed in this letter to improve the DIBR oriented view synthesis. In the spatial domain, a GWE aware view warping and blending enhancement module is developed to adaptively improve the warped views and merging the warped views from different reference viewpoints by considering and utilizing the preserved GWE. In the temporal domain, a refinement and temporal enhancement module is designed for refining and enhancing the blended view with temporal information. Experiments conducted on diverse datasets demonstrate the effectiveness of addressing the geometric warping error and using the temporal information, while further ablation study confirms the effectiveness of each proposed module.