Fast block size and mode decision algorithm for intra prediction in H.264/AVC

The H.264/AVC standard achieves high compression performance compared to existing video coding standards by employing the rate distortion optimization (RDO) technique. All possible combinations of modes are calculated to find the minimum rate distortion cost which results in significant computational complexity. In this paper, we propose a fast block size and mode decision method for intra prediction, which can effectively reduce computational burden. For fast block size decision, intra16x16 is selectively searched based on the homogeneity of inner-8x8 block of macroblock (MB). For fast mode decision, the search of intra16x16 and intra4x4 prediction modes is restricted by the similarity of the reference pixels. Moreover, by considering the similarities of the upper reference pixels and left reference pixels separately, considerable reduction of computational complexity is achieved. The results of simulations demonstrate maxima of 79% and 77% average time savings with negligible loss in PSNRY and bitrate.