A scalable fast mode decision algorithm for H.264

This paper presents a scalable fast mode decision algorithm to effectively choose the best coding mode among the 7 macroblock (MB) coding modes that could be used in H.264 video coding. The spatial and temporal complexity of the MB is analyzed to determine the order of searching in the mode decision priority queue such that the most probable mode is checked first, followed by the second most probable mode, and so on. This process is terminated as soon as the computed rate-distortion (RD) cost is below a threshold which is dependent on the complexity ratio of the current MB. By adjusting the threshold we can choose a preferred tradeoff between timesaving and quality compromise. Experimental results show that the proposed fast mode decision algorithm can drastically reduce the encoding time up to 50% with negligible loss of coding efficiency.