CNN Filter for RPR-Based SR in VVC with Wavelet Decomposition

In this paper, we propose a convolutional neural network (CNN) filter for reference picture resampling (RPR)-based super-resolution (SR) with wavelet decomposition. The proposed CNN filter takes the low resolution (LR) reconstructed frame (Rec
LR
), LR prediction frame (Pre
LR
) and high resolution (HR) RPR upsampled frame (RPR
output
) as the input for RPR-based SR. Thus, the proposed CNN filter not only learns a mapping function between LR and HR images, but also effectively removes blocking artifacts in the reconstructed frame. We adopt wavelet decomposition to make RPR
output
the same size as Rec
LR
and Pre
LR
as well as obtain the relationship between high frequency (HF) and low frequency (LF) components. To maximize feature reuse under the limited parameters, we design a residual spatial and channel attention block (RSCB) that combines residual blocks with spatial attention and channel attention to learn the weighted local information and global information in different receptive fields. Experimental results show that the proposed CNN filter achieves -8.98% and -4.05% BD-rate reductions on Y channel in AI and RA configurations over VTM-11.0_NNVC-2.0 anchor, respectively.