Cross Spectral and Spatial Scale Non-local Attention-Based Unsupervised Pansharpening Network

Pansharpening means fusing the low spatial resolution multispectral image (LRMSI) and the panchromatic (PAN) image to get the high resolution multispectral image (HRMSI). Due to the powerful feature learning ability of the deep-learning (DL), DL-based unsupervised fusion methods have been developed explosively. However, most of the fusion methods are difficult to fully explore and utilize the correct spatial and spectral correlation between the LRMSI, HRMSI, and PAN images. In addition, the CNN-dominated fusion framework is limited by its local feature learning without exploring the global feature dependency to further enhance the image feature. Therefore, to fully exploit the correct correlations between LRMSI, HRMSI, and PAN images and to explore the global feature dependency, we designed a cross-scale unsupervised fusion network (CSFNet). This network is composed of two cross spectral and spatial scale's nonlocal attention blocks to effectively fuse the LRMSI and PAN image features. And the fusion strategy is implemented by mapping the computed nonlocal similarity from the low resolution scale to the high resolution scale and outputs the reconstructed HRMSI feature. The experimental results on two datasets show that it achieves state-of-the-art performance compared to other fusion methods.