Deepsen3: Deep Multi-Scale Learning Model For Spatial-Spectral Fusion Of Sentinel-2 And Sentinel-3 Remote Sensing Images

Recently, deep learning methods that integrate image features gradually became a hot development trend in fusion of multispectral and hyperspectral remote sensing images, aka multi-sharpening. Fusion of a low spatial resolution hyperspectral image (LR-HSI datacube) with its corresponding high spatial resolution multispectral image (HR-MSI datacube) to reconstruct a high spatial resolution hyperspectral image (HR-HSI) has been a significant subject in recent years. Nevertheless, it is still difficult to achieve a high quality of spatial and spectral information fusion. In this paper, we propose a Deep Multi-Scale Learning Model (called DeepSen3) of spatial-spectral information fusion based on multi-scale inception residual convolutional neural network (CNN) for more efficient hyperspectral and multispectral image fusion from ESA remote sensing satellite missions (Sentinel-2 and Sentinel-3 images). The proposed DeepSen3 fusion network was applied to Sentinel-2 MSI (13 spectral bands with a spatial resolution ranging from 10, 20 to 60 m) and Sentinel-3 OLCI (21 spectral bands with a spatial resolution of 300 m) images. Extensive experiments demonstrate that the proposed DeepSen3 network achieves the best performance (both qualitatively and quantitatively) compared with recent state-of-the-art deep learning approaches.