All in One: A Unified Network for Hyperspectral Image Fusion

Hyperspectral image fusion (HIF) is pivotal in scientific research and engineering. With the fast development of deep learning techniques, numerous HIF algorithms have been proposed. However, traditional HIF algorithms, which are designed specifically for different input conditions, such as hyperspectral-multispectral image fusion (HMF), hyperspectral-panchromatic image fusion (HPF), and hyperspectral-multispectral-panchromatic image fusion (HMPF), cause significant inconvenience in the practical application of HIF. To overcome this issue, we propose a unified network (UniNet) that integrates HMF, HPF, and HMPF in an “all in one” manner. During testing, UniNet can perform HIF under different input conditions, enhancing its practicality. Specifically, the UniNet consists of the HMF, the HPF, and the HMPF sub-networks. Each sub-network is designed to reconstruct a high-resolution (HR) hyperspectral image (HSI) under a specific input condition. A mutual learning strategy is proposed to establish the relationships among sub-networks, enabling the transfer of spectral-spatial knowledge across different input conditions. In addition, a multiscale gradient spatial attention module (MGSAM) is designed in the network to better extract the spatial edge details of HSIs at multiple scales. Once trained, UniNet is capable of working with its sub-networks independently. Extensive experiments show that UniNet outperforms 18 state-of-the-art algorithms with five HSI datasets. Furthermore, the effectiveness of our algorithm is validated through ablation experiments.