Joint Super-Resolution and Nonuniformity Correction Model for Infrared Light Field Images Based on Frequency Correlation Learning

Super-resolution (SR) for the camera array-based infrared light field (IRLF) images aims to reconstruct high-resolution sub-aperture images (SAIs) from their low-resolution counterparts. Existing SR methods mainly focus on exploiting the spatial and angular information of SAIs and have achieved promising results in the visible band. However, they fail to adaptively correct the nonuniform noise in IRLF images, resulting in over-smoothness or artifacts in their results. This study proposes a novel method that reconstructs high-resolution IRLF images while correcting the nonuniformity. The main idea is to decompose the structure and nonuniform noise into high- and low-frequency components and then learn the frequency correlations to help correct the nonuniformity. To learn the frequency correlation, intra- and inter-frequency units are designed. The former learns the correlation of neighboring pixels within each component, aiming to reconstruct the structure and coarsely remove nonuniform noise. The latter models the correlation of contents between different components to reconstruct fine-grained structures and reduce residual noise. Both units are equipped with our designed triple-attention mechanism, which can jointly exploit spatial, angular, and frequency information. Moreover, we collected two real-world IRLF-image datasets with significant nonuniformity, which can be used as a common base in the field. Qualitative and quantitative comparisons demonstrate that our method outperforms state-of-the-art approaches with a clearer structure and fewer artifacts. The code is available at https://github.com/DuYou2023/IRLF-FSR.