Most probable mode-based fast 4 × 4 intra-prediction in H.264/AVC

The H.264/AVC uses the rate distortion optimization (RDO) method as a measure to select the optimal coding mode. However, the computational complexity for RDO calculation is increased due to the various coding modes. In this paper, we analyzes that the best mode decision for 4 × 4 intra-prediction can be affected according to the relation of most probable mode (MPM) and prediction mode. With this understanding, we propose an algorithm for fast 4 × 4 intra-prediction mode selection based on the MPM in the H.264/AVC encoder. A defined rate distortion (RD) cost, including the minimum bit for signaling the prediction modes except the MPM, is used as a threshold for fast 4 × 4 intra-prediction. Experimental results show that the proposed algorithm is capable of reducing the overall encoding time by 27–29% and the overall intra-prediction time by 35% compared to full search of the reference software, without loss of quality.