A fast mode decision algorithm for H.264/AVC inter prediction

The H.264/AVC video coding standard provides considerably higher coding efficiency than previous standards do. To achieve high performance many new tools have been introduced, however, these new features also result in a tremendous increase in encoder complexity. In this paper, we proposed a fast mode decision algorithm to reduce the complexity of inter prediction in P slices. In our algorithm for every macroblock (MB) the SKIP mode is checked first to early terminate the mode decision process, and then the temporal correlation between current MB and its corresponding MB in the previous frame is utilized to reduce the candidate modes. Of all the inter prediction modes, inter 8×8 mode has the highest complexity, so we use the motion costs of other MB modes to eliminate some of the sub-MB modes. Finally we use selective intra mode decision algorithm to check intra modes only when it is necessary. The experiment results show that the proposed algorithm provides significant complexity reduction while the degradation in video quality is negligible.