SSGT: Spatiospectral Guided Transformer for Hyperspectral Image Fusion Joint With Cloud Removal

The hyperspectral image (HSI) has poor spatial resolution and is vulnerable to cloud, which limits its application in practical tasks. However, current hyperspectral and multispectral image fusion algorithms do not consider the cloud contamination problem. In view of this situation, we propose a novel dual-branch fusion framework to improve spatial resolution of HSI while restoring the details of areas covered by clouds. The network extracts multiscale and multilevel spatiospectral features through the dual-branch encoder–decoder structure to focus on local information, and acquires the nonlocal similar relationship by transformer to aggregate global information of features for better reconstruction. To produce a more natural image, the spatiospectral gradient loss with L1 loss of reconstructed image is introduced to guide network training. It can be seen from the cloud-free and cloud-contaminated experiment results that the proposed method achieves the best results in both visual and accuracy evaluation, compared with the recent state of the arts.