Sub-Frame Layer Rate Distortion Optimization for High Efficiency Video Coding

This paper proposes a sub-frame layer RDO algorithm to further improve coding performance of low-delay hierarchical prediction structure in High Efficiency Video Coding. In the proposed algorithm, each frame is segmented into two sub-frames according to motion intensity, and a sub-frame layer is taken as a basic optimizing unit. An inter-frame dependency model is proposed to quantify inter-frame dependency of a frame or a sub-frame. The proposed inter-frame dependency model is composed of two parts, Laplace distribution based direct inter-frame dependency model and distortion propagation chain. The direct inter-frame dependency model is used to evaluate dependency caused by direct reference, and the distortion propagation chain shows the relationship between direct reference and indirect reference. With quantified inter-frame dependency, Lagrange optimization is then used to solve the optimal $\lambda $ -QP pair for each sub-frame layer in frame layer 1 (L1) to frame layer 4 (L4). For frame layer 0 (L0), additional L0 frames are adaptively inserted according to inter-frame dependency of the previous L0 frame, and the optimal $\lambda $ -QP pair for each sub-frame in L0 is solved using Lagrange optimization. When peak signal to noise ratio is used as quality criterion, the proposed algorithm achieves 7.0% and 7.0% bit-rate reduction with lowdelay_P_main (LDP) and lowdelay_main (LDB) configurations respectively; when structural similarity is used as quality criterion, the proposed algorithm achieves 13.3% and 13.1% bit-rate reduction with LDP and LDB configurations respectively.