Nonlocal Deep Unfolding Pansharpening Method Based on Degradation Kernel Estimation

Variational optimization (VO) pansharpening method proposes a model for optimizing the energy function by delineating the acquisition process of remote sensing images. However, traditional VO methods face challenges in solving the operators. Currently, some model-driven methods alleviate this problem by employing deep unfolding techniques. However, most model-driven approaches do not take into account the unknown and variable degradation process inherent in remote sensing images when solving the optimization model. They typically utilize a fixed Gaussian downsampling operator or a two-layer convolutional neural network module to directly simulate the degradation process during model unfolding, which leads to models that lack sufficient generalization. Therefore, this article proposes a pansharpening method based on deep nonlocal unfolding with degradation kernel estimation. Specifically, we expand the iterative process of solving the energy function into three modules: degradation kernel estimator, image generator, and three-branch prior network. First, the degradation kernel estimator is employed to fit the real degradation process of the image and optimize the generation of an adaptive degradation kernel. Simultaneously, we incorporate local priors, nonlocal spatial priors, and nonlocal spectral priors into the three-branch prior network to adaptively capture local and nonlocal prior features. These prior features are then fused with the generated image through residual connections to approximate the real remote sensing image. Experimental results on datasets from two different types of satellites demonstrate the superiority of our approach.