Transform Kernel Selection Strategy for the H.264/AVC and Future Video Coding Standards

In this paper, we propose a new discrete cosine transform (DCT)-like kernel IK(5, 7, 3) and revitalize another DCT-like kernel IK(13, 17, 7) for the transform coding process of hybrid video coding. Making use one of these kernels together with the H.264/AVC kernel IK(1, 2, 1), we are able to design new multiple-kernel schemes which give better coding performance over that of the conventional approaches. All these schemes make use of the adaptive kernel mechanism at macroblock-level (MB-AKM), which requires heavy computation during the encoding process. We subsequently discovered that a rate-distortion feature extracted from a pair of kernels gives an intrinsic property that can be used to select a better kernel for a two-kernel MB-AKM system. This is a powerful tool with theoretical interest and practical uses. In order to reduce computation substantially, we make use of this tool to make an analysis and design of a frame-level adaptive kernel mechanism and come up with a simple solution that the kernel IK(1, 2, 1) be used for I-frames and P-frames and the kernel IK(5, 7, 3) be used for B-frames coding. This proposed frame-based AKM gives similar, or even better, performance as the proposed macroblock-based AKM. Furthermore, it substantially reduces computation and certainly gives a good improvement in terms of the PSNR and bitrate compared to those obtained from the H.264/AVC default scheme and other MB-AKM schemes available in the literature.