Multimodal Sensors Image Fusion for Higher Resolution Remote Sensing Pan Sharpening

Panchromatic (PAN) sensors and multispectral (MS) sensors are usually applied to remote sensing tasks. Pan sharpening is an effective image fusion method based on multimodal sensors. In pan sharpening, high-resolution (HR) PAN images from PAN sensors and corresponding low-resolution MS images from MS sensors are used to obtain the HR MS images. Normally, the ideal pan-sharpening image is a fused MS image with the same resolution as the PAN image. We hope to obtain the pan-sharpening images with higher resolution, which will not only preserve spatial structure information and spectral information from PAN images and MS images but also achieve super-resolution reconstruction. For this purpose, a novel pan-sharpening framework, called multilevel and multiscale fusion network (MLMSFN), is constructed. To our knowledge, our framework is the first work to achieve beyond the limit of existing image resolution in pan sharpening. We evaluate the effectiveness of our designed method, and the experiments testify that our method can obtain higher resolution pan-sharpening images.