A Unified Framework for Super-Resolution via Clean Image Prior

Super-resolution (SR) methods aim to generate a clean, high-resolution (HR) image from a low-resolution (LR) image containing unknown degradations. Previous super-resolution approaches typically follow a two-stage framework, estimating degradation-related information, and then, restoring HR images based on the estimated information. However, such approaches lack generalization ability to various degradations and have difficulties in fully leveraging the information of degradation. This paper deals with the SR problem from a new perspective: learning clean image prior instead of degradation-specific information. Specifically, we introduce the clean feature extraction (CFE) module that generates clean image features from degraded inputs. Subsequently, the efficient frequency decomposition SR (EFDSR) module is applied to produce clean HR images from the extracted clean features. Our EFDSR module is designed to decompose feature into different frequency bands and focus on more informative components, resulting to produce fine SR results. Furthermore, we introduce a two-phase training pipeline for our framework, allowing the joint training in an end-to-end manner. Experimental results show that our SR framework outperforms state-of-the-art (SOTA) performance not only on LR images with unknown blur but also for various degradations such as noise and JPEG compression artifact, e.g., achieved 29.91dB PSNR on DIV2K validation dataset (with random anisotropic Gaussian blur), exceeding the previous SOTA method by 0.20dB. We further show that our SR framework produces the closest results to the true HR images, without unpleasant distortion, for severely degraded images by fully learning the mapping from degraded LR images to clean HR images.