Advanced H.264/AVC-Based Perceptual Video Coding: Architecture, Tools, and Assessment

The characteristics of the human visual system may be further exploited in lossy video coding to improve the video compression efficiency beyond the state-of-the-art H.264/AVC standard. Although the literature is rich in solutions to model the human visual system characteristics, the performance and real benefits brought by these models have not been fully integrated and assessed yet. Moreover, the rate-distortion (RD) performance is usually measured by means of methodologies that do not account for the implicit variability of the observers when rating the video quality. In this context, the novelty brought by this paper is threefold: first, it proposes novel perceptual video coding tools, notably decoder side just noticeable distortion (JND) model estimation to perceptually allocate the available rate with the finest level of granularity while avoiding the extra rate associated to coding the varying quantization steps. Second, it proposes an integrated, powerful H.264/AVC-based perceptual video coding architecture embedding a state-of-the-art JND model based on spatio-temporal human visual system masking mechanisms; this model is exploited for both the aforementioned rate allocation as well as to perceptually weight the distortion used in the motion estimation and RD optimization. Finally, it proposes a relative assessment methodology to measure the RD performance of a perceptual video codec (PVC) with respect to another codec taken as reference. The methodology considers the implicit observers variability when rating video quality which leads to a nonlinear sensitivity of the objective metrics used for quality assessment. The obtained RD performance, measured according to this methodology, shows an average bitrate reduction of up to 30% when the proposed PVC is compared with the H.264/AVC High profile at the same objective quality level. Moreover, the proposed perceptual codec outperforms an alternative perceptual codec recently published in the literature.