Content-Aware Motion Compensated Temporal Filter for Video Coding

Video coding achieves efficient compression by exploiting the spatial and temporal correlations within the video signal. However, the noise in the source signal corrupts such correlation and impairs the coding performance. Motion compensated temporal filter (MCTF) [1] is a pre-processing tool that removes certain noise from the source signal, thereby enhancing temporal correlations among adjacent frames. Although numerous efforts have been made to optimize MCTF, MCTF still lacks flexibility in filtering for diverse video content, and its filtering efficiency is still limited. In this paper, we propose the Content-Aware MCTF method (CAMCTF) to enhance the filtering adaptability of MCTF for diverse video contents. The CAMCTF adaptively adjusts the filtering block sizes based on the Sum of Square Error (SSE) and Motion Vector (MV) information calculated during the motion estimation (ME) process in MCTF, and offset weighting method is applied to improve the prediction quality of filtering blocks. Performance was evaluated on top of Versatile Video Coding (VVC) reference software VTM-23.4. The VVC Common Test Conditions (CTC) [2] with QPs 22, 27, 32, 37 are used. As shown in Table 1, CAMCTF achieves an overall of 1.07% and 0.79% luma BD-rate gains in Random Access (RA) and Low Delay (LD) configurations, respectively. The encoder complexity is 112% and 110% for RA and LD configurations, respectively, without decoder complexity increasing.