Deep Explainable Content-Aware Per-Scene Video Encoding

In the era of increasing video streaming, optimizing content delivery and finding a good trade-off with minimal effort between quality and size is a critical task. However, varying sensitivities to bandwidth loss across different video content types demand a robust and customized approach to encoding parameters within longer videos containing various scenes. This paper presents a promising novel approach to video encoding, leveraging machine learning to enhance content delivery efficiency, and offers insights into the model's decision rationale. Traditional methods of estimating quality loss through multiple test encodes and interpolation are computationally intensive. To address this challenge, we introduce “Deep Explainable Content-Aware Per-Scene Video Encoding”, a machine learning-based approach to video encoding quality prediction given the video scene and encoding parameters. Moreover, we integrate explainable artificial intelligence to enhance our understanding of the model's decision-making process. We have encoded various video scenes using traditional methods and trained our model to learn the relationship between quality loss and specific video contents. We aimed to have our model consider how a specific video scene changes over time; thus, we employed long short-term memory (LSTM) neural networks for predicting quality loss. Our preliminary results demonstrate good accuracy and efficiency, as well as the content awareness of the model.