Multi-QP Rate Distortion Optimized Quantization Using Deep Learning

RDOQ (Rate Distortion Optimized Quantization) is an efficient encoding tool that can be used with several codecs such as H.264/AVC, H.265/HEVC or AV1. Although this algorithm can significantly reduce the bit rate, its complexity is a limitation for video coding hardware solutions. Studies have succeeded in simplifying RDOQ allowing its integration into HM, the reference software implementation of H.265/HEVC. However, its iterative and sequential behavior does not allow an efficient hardware implementation. With the advent of machine learning, neural networks have been introduced to mimic the RDOQ algorithm in a parallel way. Previous proposed Deep-Learning based RDOQ frameworks still need to be trained for each Quantization Parameter (QP) which is a huge limitation for hardware implementation. To address this issue, we propose a Multi-QP Deep Learning based RDOQ, trained with data extracted from 4 QPs and achieving targeted performances with QP ranging from 22 to 37. Our multi-QP model almost reaches HM-RDOQ performance in terms of BD-Rate savings.