Depth video coding for free viewpoint video oriented to the synthetic view perceptual quality

In this paper we propose a novel depth encoding algorithm based on the synthetic view perceptual quality optimization. In Free Viewpoint Video, depth sequences are never shown to the observer but they are used for the view synthesis. Due to the compression quality losses, the depth distortion generates errors in the synthetic view pixel position. We propose to encode the depth sequences by a novel approach, divided into two parts. The first one is a rate distortion optimization method based on the minimization of the synthetic view pixel position error. The second one is an extension of the first one and improves the performance taking into account the perceptual distortion, evaluated by exploiting the Human Visual System characteristics described by the Just Noticeable Distortion. However, the Just Noticeable Distortion is a model designed for the traditional video, hence we also propose a Just Noticeable Pixel Displacement evaluation method which, considering the synthetic view perceptual distortion, is able to estimate when an error in the synthetic view pixel position is not noticed. The results show how the proposed algorithm not only improves the synthetic view quality measured by the Video Quality Metric (highly correlated to the subjective quality) but also the objective quality measured by the PSNR, with an achieved improvement of 0.3 dB with a corresponding bit saving of 13%.