Frame Rate Up-Conversion for Depth-Based 3D Video

A novel frame rate up-conversion (FRUC) scheme for depth-based 3D video is proposed. Differing from the existing conventional FRUC methods which are designed for two dimensional (2D) video only, the proposed method is designed for depth-based 3D video, which increases the frame rate of both color sequences and its associated depth maps by jointly considering the image intensity, depth information, and spatial-temporal correlations. The proposed method contains motion estimation, block irregular segmentation (BIS), depth-constrained motion vector post-processing, forward and backward motion compensation, and edge-preserved combination. Experimental results show that the intermediate color and depth images interpolated by our proposed method provide a good image quality both objectively and subjectively. Moreover, compared with conversional FRUC methods, the color and depth map sequences up-converted by the proposed method are more suitable for further virtual view synthesis.