AMSFusion: An Adaptive Multi-Scale Infrared and Visible Image Fusion Network Based on Attention Mechanisms

The goal of image fusion is to generate a new image that incorporates all the high-quality features from source images, such as the salient features of infrared images and the texture details of visible images. Existing image fusion methods primarily focus on deep feature extraction, often neglecting the importance of shallow features. Furthermore, fusion strategies that rely heavily on human intervention can lead to the loss of feature information and limit deep learning performance. To address these issues, we propose an adaptive multi-scale fusion network based on attention mechanisms, named AMSFusion. The autoencoder component of our method is based on the U-Net architecture, with different modules designed to process shallow and deep multi-scale features, thereby improving feature processing quality while maintaining computational efficiency. The fusion network component employs a channel-spatial attention block (CSA) to assign appropriate weights to heterogeneous features and applies a shifted window attention and convolution mix transformer (SWACmixT) to fuse them, enhancing the network’s ability to adaptively fuse features at different scales. In our implementation, the autoencoder and fusion networks are trained separately, with distinct loss functions for each, further enhancing the method’s capabilities in feature encoding, decoding, and fusion. Qualitative and quantitative experiments on multiple datasets demonstrate the superiority of our method compared to state-of-the-art algorithms. Additionally, experiments on object detection tasks validate that our method effectively promotes high-level computer vision tasks.