Deep Convolutional Neural Network for Decompressed Video Enhancement

Block-wise intra/inter prediction, transformation and quantization used in block-based hybrid video coding will inevitably result in blocking artifacts, especially at the low bit rate. To address this problem, this paper employs a deep convolutional neural network (CNN) to approximate the reverse function of video compression, motived by the great success of deep learning in computer vision fields recently. The proposed method establishes an end-to-end mapping, represented as the CNN, which takes the decompressed frame as input and outputs the enhanced one. Employing numerous sequences compressed by H.264 and HEVC reference software, the proposed CNN learns the connections between the lossy frame and the original one in an implicit way under different quantization parameters (QP). Figure 1 shows the architecture of our CNN and the pipeline of the network training. We build our network with convolution layers and ReLU layer and the weights and biases of all the convolution layers in our model are updated by minimizing the loss using stochastic gradient descent with the standard backpropagation. We implement the CNN as a post-loop deblocking filter and explore varying CNN parameters for different QPs. Various experimental results demonstrate that the proposed method is able to significantly improve the quality of enhanced frames in terms of both objective and subjective criterions.