Motion estimation with entropy coding considerations in H.264/AVC

In many modern video encoders, motion estimation is usually formulated as a Lagrangian cost function that balances motion prediction accuracy and the rate to transmit motion vectors. In H.264/AVC video encoders, including the H.264 reference software, in an effort to reduce computational complexity, the motion vector rate is often estimated using a look-up table based on universal variable length coding regardless of the entropy coding method used. However, the estimated rate could be different from the actual rate used for encoding. In this paper we present an empirical study on the motion vector rate estimation mismatch caused by different entropy coding assumptions and discuss its impact on motion estimation and coding efficiency. We also present a more accurate motion vector rate estimation method for encoding with the context adaptive binary arithmetic coding (CABAC) method while still maintaining low computational complexity. The proposed motion vector rate estimation can provide up to 2.26% in rate savings for low bit rate applications when all modes are considered for the encoding. Results for single mode encoding, which can be useful for fast encoding applications, show that we can achieve benefits of up to 27.15% and 7.39% in rate savings for low and medium to high bit rate applications respectively.