Adaptive weighted distortion optimization for video coding in RGB color space

In this paper, we propose an adaptive weighted distortion optimization algorithm for improving the coding performance of video represented in RGB color space. The YCbCr is the color space, which is usually adopted in video coding. And with 4:2:0 planar format, it generally uses the ratio 6:1:1 metrics objective peak signal-to-noise ratio (PSNR) for quality assessment. In this paper, we still use 6:1:1 combined PSNR as a universal criterion to optimize video coded in RGB domain and measure the final quality of decoded video as well. The new distortion metric in RGB coding is derived from the optimal target, and the modification is only applied on the encoder side. The proposed algorithm achieves about 26% bit-saving on average and maximum 44% bit-saving in HEVC range extensions reference software HM12.0 RExt4.1. Additionally, the visual quality of video coded by the proposed algorithm is also improved.