A Video Compression Framework Using an Overfitted Restoration Neural Network

Many existing deep learning based video compression approaches apply deep neural networks (DNNs) to enhance the decoded video by learning the mapping between decoded video and raw video (ground truth). The big challenge is to train one well-fitted model (one mapping) for various video sequences. Different with the other applications such as image enhancement whose ground truth can only be obtained in the training process, the video encoder can always get the ground truth which is the raw video. It means we can train the model together with video compression and use one model for each sequence or even for each frame. The main idea of our approach is building a video compression framework (VCOR) using overfitted restoration neural network (ORNN). A lightweight ORNN is trained for a group of consecutive frames, so that it is overfitted to this group and achieves a strong restoration ability. After that, parameters of ORNN are transmitted to the decoder as a part of the encoded bitstream. At the decoder side, ORNN can perform the same strong restoration operation to the reconstructed frames. We participate in the CLIC2020 challenge on P-frame track as the team "West-World".