Loading [a11y]/accessibility-menu.js
Supervised Detail-guided Multi-scale State Space Model for Pan-sharpening | IEEE Journals & Magazine | IEEE Xplore

Supervised Detail-guided Multi-scale State Space Model for Pan-sharpening


Abstract:

Pan-sharpening reconstructs the high resolution multispectral (HR-MS) image from its corresponding panchromatic (PAN) image and low-resolution multispectral (LR-MS) image...Show More

Abstract:

Pan-sharpening reconstructs the high resolution multispectral (HR-MS) image from its corresponding panchromatic (PAN) image and low-resolution multispectral (LR-MS) image. However, existing deep learning-based pan-sharpening methods typically suffer from three challenges: (1) the vanilla LR-MS image upsampling employed by them fail to consider domain knowledge, thereby disregarding crucial information; (2) while remote sensing images exhibit multi-scale complex land features, existing methods fail to fully exploit crucial multi-scale spatial information, i.e., scale transformation layers within their models are not effective; and (3) existing CNN and transformer-based pan-sharpening backbones are constrained by inherent local receptive fields or quadratic computational complexity, making them difficult to balance their effectiveness and efficiency. To address these issues, we propose a novel Supervised Detail-guided Multi-scale State Space Model for Pansharpening, namely SDMSPan. Our SDMSPan consists of three Residual State Space Modules (Res-SSMs) that are responsible for handling image information at three spatial scales, where each Res-SSM aims to model both local and long-range dependencies between PAN and LR-MS images at a specific spatial scale with lower computational cost. Between each pair of Res-SSMs, a novel Detail-Guided Upsampling Block (DGUB) is proposed to apply spatial details of the PAN image to guide effective and taskaware intermediate feature upsampling, where a novel multi-scale intermediate spatial-spectral supervision strategy is also proposed to supervise the training of every DGUB. Experimental results demonstrate that our proposed approach significantly outperforms other state-of-the-art methods in performance. Our code is provided at https://github.com/zhaomengjiao123/SDMSPan.
Page(s): 1 - 1
Date of Publication: 23 December 2024

ISSN Information:

Funding Agency: