Rate-distortion optimized frame level rate control for H.264/AVC

In this paper, a rate-distortion optimized frame level rate control algorithm is presented for H.264/AVC. To improve the performance in both distortion and rate, two techniques are developed. First, an adaptive frame layer rate-distortion optimization technique is included into the rate control module so that the average distortion is decreased. Second, the mapping from the bit budget to the quantization step is well considered at frame level by an accurate rate model. Consequently, a more precise control over the rate is achieved. Simulations show that the proposed rate control outperforms both the coding with fixed quantization step size (Fixed-QP) and the recommended rate control JVT-G012 in the H.264/AVC reference software. Compared with Fixed-QP, a gain up to 1.04 dB is observed and the maximal rate mismatch for seven sequences is only 0.61%. For JVT-G012, an improvement up to 0.42 dB is obtained while the average rate mismatch is reduced by 70%.