SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

Pansharpening, which attempts to obtain a high-resolution multispectral (HR-MS) image by fusing a panchromatic (PAN) image with a low-resolution multispectral (LR-MS) image, is a critical yet difficult remote sensing image processing task. In this study, we present a novel convolution operation, self-guided spatial-channel adaptive convolution (SSCAConv), for pansharpening. Unlike the reported adaptive convolutions that only focus on spatial details, our SSCAConv also considers channel specificity by generating an individual convolution kernel for each channel patch according to its own content and supplements the interchannel information by introducing a global bias. We further apply the designed SSCAConv to a simple residual network architecture to construct the image fusion network (SSCANet). Experimental results show that SSCANet outperforms state-of-the-art (SOTA) pansharpening algorithms and achieves better generalization ability with fewer parameters. In addition, our network also yields the best results when extended to the hyperspectral image super-resolution (HISR) problem. The code is available at https://github.com/Pluto-wei/SSCAConv.