A Novel Fast Inter-mode Decision Algorithm for H.264/AVC Based on Motion Estimation Residual

This document proposes a fast inter-mode decision method based on the residual of motion estimation for H.264/AVC. There are 7 different MB modes in inter-mode coding with a SKIP mode, as well as 9 modes of 4 times 4, 8 times 8 and 4 modes of 16 times 16 in intra-mode coding for P-slices coding, which brings drastic computational complexity to the encoder. To solve this problem, we suggest a novel fast inter-mode decision method using the variance and SAD of residual block. Our method computes the motion estimation residual block's variance and sum of absolute value (i.e. SAD) to decide the best mode of current MB, and computes residual sub-blockpsilas sum of absolute value to decide sub-MBpsilas mode. Experimental results show that our method can save 30%-50% encoding time on PC platform without significant PSNR degradation or bitrate increasing.