Optimal quadtree for variable block size motion estimation

In this paper, we present a new approach to find the best quadtree for variable block size (VBS) motion estimation for video coding. In this approach, the quantizer which encodes the prediction residual is also jointly optimized in a rate-distortion sense. The desired balance of the bit allocation between the motion vector coding and the residual error coding is achieved by a Lagrange multiplier approach for the joint optimization problem. A fast tree optimization technique is used to reduce the computational complexity of the algorithm so that it can be run in realtime in a appropriate hardware.