Rate Estimation, Using Forward Adaptive Quantization: H.264 Fast Intra Mode Selection at High Data Rates

This paper presents a fast method to estimate the required bits to encode a block of video data. The proposed method uses forward adaptive quantization (which uses the current sample information, not available at the decoder, to scale the sample value before and after quantization) to estimate the actual rate-distortion cost. Rate estimation has many applications in the art of video coding, where a coding decision should be made based on the optimality of a rate-distortion pair. As an example this method has been considered for fast rate-distortion optimized mode selection within H.264/AVC video coding standard. It is shown that, compared to other rate estimation methods, the proposed method more accurately predicts the optimal mode identical to the one chosen by a Lagrangian cost function based on the actual rate and distortion. The computational complexity of forward quantization is much lower than that of conventional entropy coders which makes it possible to perform the motion estimation with rate-distortion criterion as opposed to simple sum of absolute values of residual signal, which increases the efficiency of predictive coding.