Pansharpening of Hyperspectral Images with Detail Guided Feature Modulation

Pansharpening of hyperspectral image (HSI), which makes use of the detail information contained in the high-resolution panchromatic (HR-PAN) image to sharpen the low-resolution HSI (LR-HSI), is an essential technology to enhance the spatial resolution of HSI. In this paper, we propose a detail guided feature modulation residual network (DGFM-Net) to address the HS pansharpening problem, which is able to effectively integrate details extracted from the PAN image into the pansharpened result. Specifically, we elaborately design a novel feature modulation (FM) module with the guidance of PAN detail information to modulate HSI features flexibly and incorporate PAN details adaptively. The modulated features are then fed to the residual reconstruction (RR) block to recover the difference between the upsampled HSI and the HR-HSI by efficient residual learning. Finally, the upsampled HSI is combined with the estimated residual HSI to produce the desired HR-HSI. Experiments on the Pavia Center data set confirm that the proposed DGFM-Net outperforms several state-of-the-art HS pansharpening methods.