Depth-based weighted bi-prediction for video plus depth map coding

Video with depth is attracting attention as a promising approach to future 3D video. Unlike conventional video coding technologies, video with depth should take the correlations between video and depth into account. This paper proposes a novel bi-prediction scheme for video coding that exploits the structure similarity between video and depth. In the proposed scheme, a prediction image is generated by the pixel-wise weighted average of motion compensated reference images, and weights are decided by considering pixel similarities in the depth domain. The proposed pixel-wise weighted bi-prediction offers implicit depth-based block partitioning with consideration of the pixel correlations between predicted and reference blocks. Compared to using explicit, arbitrary shaped block partitioning, the proposed method offers better prediction for non-rigid objects, and requires no increase in motion information. Experiments show that the proposed method can achieve up to 17% bitrate reduction for bi-predictive frames and about 8% on average for 6 sequences relative to the existing biprediction scheme.