Rate-Distortion and Complexity Joint Optimization for Fast Motion Estimation In H.264 Video Coding

H.264 video coding standard offers several coding modes including inter-prediction modes that use macroblock partitions with variable block sizes, Choosing a rate-distortion optimal mode among these possibilities contributes significantly to the superior coding efficiency of the H.264 encoder. Unfortunately, searching for optimal motion vectors of each possible subblock incurs a heavy computational cost, In this paper, in order to reduce the complexity of integer-pel motion estimation, we propose a rate-distortion and complexity-joint optimization method that selects for each MB a subset of partitions to evaluate during motion estimation. This selection is based on simple, measures of spatio-temporal activity within the MB. The procedure is optimized to minimize mode estimation error at a certain level of computational complexity. Simulation results show that the algorithm speeds up the motion estimation module by a factor of up to 20 with little loss in coding efficiency.