H.264 visual perceptual coding in uniform analyzing and encoding framework

The perception of Human Visual System (HVS) for the video scene is selective. Different regions in the video scene have distinctive levels of visual importance. In this study, we present a novel H.264 visual perceptual video coding (VPVC) method in a uniform analyzing and encoding framework, which can allocate bit and computation resources effectively. The framework presented in this work consists of a visual perception model, a H.264 perceptual encoder and a corresponding sharing channel of information. The visual perception model, composed of motion perception, texture perception and spatial position perception sub-models, can compute the visual perception map (VPM) by fusing these spatiotemporal visual features. Visual perception results of HVS for various regions can be simulated well by VPM. The side encoding information of H.264 encoder, including motion vectors (MVs) and sum of absolute differences (SADs), is applied as input features for motion perception sub-model. A novel VPVC method is proposed based on the VPM and the global motion type of video scene. Using an adaptive frequency coefficient suppression technique and a novel encoding strategy, the optimal bit resource allocation is achieved by classifying video scene based on the VPM. In order to allocate computation resource effectively in VPVC method, the relation between optimal encoding mode and image features at video scene level is experimentally analyzed. As a result, a fast and effective H.264 mode analysis algorithm is deduced. When compared with the conventional H.264 coding method, our results on four video sequences show that the proposed method can obtain a high PSNR gain up to about 2.0 dB for visual important regions and decrease about 38% of total encoding time on average.