Differentiable VMAF: A trainable metric for optimizing video compression codec

Video Multi-method Assessment Fusion (VMAF) is a widely used objective evaluation metric that has shown a stronger correlation with human visual system than other metrics. Some studies have already integrated VMAF as a measure of perceptual quality in the rate-distortion optimization of hybrid codecs. However, VMAF has long been considered non-differentiable, limiting its application in learning-based video compression. In this work, we successfully achieve a differentiable VMAF through mathematical approximations. Our implementation of differentiable VMAF maintains the performance of the original VMAF while ensuring accurate gradient calculations. By integrating it into the loss function for optimizing learning-based video compression, our experimental results demonstrate that this approach significantly enhances VMAF performance and improves perceptual quality at all bitrates.