Pan-DUSM: Deep Unfolding Network With State Space Model for Pansharpening

The purpose of pansharpening method is to combine the spectral-spatial information from both the multispectral images and the panchromatic (PAN) images to obtain the high-resolution multispectral (HRMS) image. With the rapid development of CNN and Transformer architectures in recent years, pansharpening methods have undergone significant advancements. However, most of the existing algorithms suffer from low interpretability, limited global representation capability, and quadratic computational complexity. To address these challenges, this study focuses on developing an algorithm with strong interpretability and efficient long-term dependency modeling. Drawing inspiration from contemporary variational optimization (VO) and selective state space models, we propose a network with strong interpretability and efficient global information modeling named Pan-DUSM in this article. The network consists of an encoder and a decoder. The encoder leverages a VO model to enhance overall representation by incorporating deep prior knowledge as implicit regularization, though its feature extraction capacity is inherently limited. To address this, we introduced the residual state space model module in the decoder section, which is adept at extracting long-range feature information. In addition, the local effective feature extraction module is introduced to refine local image details, while the Concat-SSM module facilitates efficient interaction between multiscale global features and local details, thereby improving the reconstruction of image information. We compare our method with current state-of-the-art pansharpening methods on QB, GF2, and WV3 datasets. Experimental results show that our proposed model achieves the best performance.