CAFE: A Cross-Attention Based Adaptive Weighting Fusion Network for MODIS and Landsat Spatiotemporal Fusion

Dense medium-resolution images play an important role in time-series geoscience applications. However, due to technical limitations, remote sensing imaging systems inevitably trade off temporal frequency and spatial swaths, resulting in difficulties to acquire images simultaneously with high spatial and temporal resolution. To overcome this limitation, under the framework of residual learning, we propose a cross-attention-based adaptive weighting fusion network (CAFE) for MODIS-Landsat spatiotemporal fusion to generate dense medium-resolution images. Based on the cross-attention mechanism, we propose multichannel separated cross-attention (MSCA) and full-feature joint cross-attention (FJCA) blocks to enhance spatial resolution and retain spectral signatures from the perspectives of band-wise processing and full-feature joint processing, respectively. The adaptive temporal difference weighting mechanism (ATDWM) is proposed to improve the ability to capture dynamic land surface changes. Besides, we employ an adaptive fusion loss function to constrain the network training. Experimental results indicate that the developed method is superior to several existing algorithms in terms of visual evaluation and quantitative evaluation and it can generate high-quality fusion results in scenarios of both subtle and dramatic temporal changes. Codes are available at
https://github.com/LiupengLin/CAFE
.