A Generic Multicorrespondence Matching Framework for Reference-Based Image Super-Resolution

Image super-resolution (SR) plays an important role in vision-based measurements, and reference-based image super-resolution (RefSR) is one such SR technique that enhances the resolution of a low-resolution (LR) image using an external high-resolution (HR) reference image. However, most existing RefSR methods search for the best-matching correspondence in the reference image, typically a single patch or pixel, to facilitate SR reconstructions, leading to reference underuse and misuse issues, since there are several texture details in different positions of the reference image for pairing with each image pixel or patch in the LR image. To this end, a novel generic framework for multicorrespondence matching is proposed in this article, which aims to identify multiple similar correspondences within the reference image for an LR patch or pixel. In our framework, multiple similar features from the reference image are exploited and aligned, and then they are all aggregated with the LR feature. The contribution of each similar feature from the reference image to the SR reconstruction process is determined by its similarity with the corresponding LR feature. Our designed framework facilitates the search for multiple similar correspondences within the same reference image. This approach enables us to fully harness the external content and textures present in the reference image, effectively mitigating the issues of underuse and misuse of the reference image. Furthermore, our innovative framework seamlessly integrates with existing RefSR models. It effortlessly incorporates existing RefSR models and generates multicorrespondence matching versions of them, leading to notable enhancements in their SR performance. We empirically show that existing state-of-the-art RefSR methods can be consistently improved by our framework, further improving the performance of these methods. Extensive quantitative and qualitative experiments validate the effectiveness of the proposed framework and demonstrate its superior performance. The code of the proposed method is available at
https://github.com/ModestTony/MCMSR
.