SUR-Driven Video Coding Rate Control for Jointly Optimizing Perceptual Quality and Buffer Control

Rate control plays an important role in video coding and has attracted lots of attention from researchers. However, the problems of human visual experience and buffer stability still remain. For scenes with drastic motions, parts of distortions can be masked due to the limitation of the Human Visual System (HVS), while buffers tend to suffer more overflow and underflow cases from the fluctuating bits. In this paper, we propose a novel joint rate control scheme, which is composed of the proposed SUR-based perception modeling and the proposed SUR-based Perception-Buffer Rate Control (PBRC), for HEVC to maximize human visual perception quality while preventing the underflow and overflow of buffers. First of all, to effectively model human visual quality, we introduce the perception-related Satisfied-User-Ratio (SUR) metric into the rate control process. Secondly, a time-efficient video quality prediction method called Fast Visual Multimethod Assessment Fusion (VMAF) Quality Prediction (FVQP) is designed for the generation of SUR curves within an affordable computational complexity. Thirdly, a dual-objective optimization framework is established. By jointly conducting perception modeling and PBRC, we can flexibly adjust the optimization priority between human visual quality and buffer stability, and thus the quality of achieved reconstructed videos can be effectively improved because of the decrease in frame skipping. Experimental results demonstrate that the proposed joint rate control scheme improves the human visual experience when considering frame skipping and more effectively stabilizes buffer stability than existing methods.