Dual edge-confined inpainting of 3D depth map using color image's edges and depth image's edges

Currently, most 3D formats adopt the approach of 2D plus depth map. Thus, many papers discuss depth maps, including how to inpaint them. When inpainted, the main focus is on edge areas because poor edges in depth maps will result in two problems: (1) There will be holes in 3D images after image synthesis; (2) When synthesized for other views, an edge mismatch may create some visual defect at the occlusion part of the images. In this paper, we propose a method to enhance the consistency between a color image and its depth map. With 2D color image's edges and depth map's edges, the proposed method finds the areas where the edges do not match, and uses a dual edge-confined inpainting technique to inpaint the edge-mismatched areas. The inpainting technique exploits the relationship between edges and objects in color image and depth map, and conducts inpainting directionally, where a depth map is inpainted at the inconsistent part between a depth map and its corresponding color image to improve the quality of the depth map. The experimental results show that the proposed method enhances the edge consistency between the depth map and the corresponding color image. In addition, following traditional deblocking process in image coding, the proposed method increases the PSNR value of depth map for QP less than 30, and achieves a comparable performance to that obtained by using the trilateral filter plus SD algorithm which also uses color image and depth map information.