Coding and rendering enhancement of multi-view video by color correction

Multi-View Video (MVV) is captured from different viewpoints with multiple cameras, where substantial inconsistencies of illumination and color are often observed between the different views. These color mismatches can reduce significantly the compression efficiency and the performance of image-based rendering algorithms. Therefore, we propose a preprocessing method based on the Histogram Matching (HM) algorithm for correcting these color discrepancies in MVV. To consider occlusion problem in MVV, we perform the HM algorithm only on a set of common regions across views. These regions are defined using an invariant feature detector (SIFT), followed by RANSAC (Random Sample Consensus) algorithm to increase the matching robustness. In addition, to maintain temporal correlation, HM algorithm is applied on a temporal sliding-window. The choice of the color reference is also addressed in this paper and is carried out automatically in a more adaptive way. Experimental results show that the proposed method increases coding efficiency with gains of up to 1.1 dB and 2.2 dB for the luminance and chrominance components, respectively. Moreover, once correction is performed, the color of both real and rendered views is harmonized and looks very consistent as a whole.