Coding Efficiency of the DCT and DST in Hybrid Video Coding

Standardized hybrid video coding algorithms, e.g., HEVC, apply intra frame prediction or motion compensated prediction and subsequent integer approximated DCT or DST transform coding. The coding efficiency of the transforms depends on the statistical moments and probability distribution of the input signals. For a Gaussian distribution, the DCT always leads to a data rate reduction. However, for Laplacian distributed prediction errors, the transforms sometimes increase the data rate. This paper presents a theoretical analysis, which explains the reason for an increase of the data rate, which is due to the generation of higher statistical moments of the coefficients by the DCT or DST in the case of Laplacian distributed input signals. The data rate can increase by up to 0.10 bit per sample for blocks with low correlation. For screen content, in about 20% of the blocks, the transform increases the data rate. By skipping the transform for these blocks, HEVC achieves a 7% data rate reduction.