Infrared and Visible Image Fusion Algorithm Based on Improved Residual Swin Transformer and Sobel Operators

Infrared and visible image fusion is of great significance in many applications, such as automatic navigation, security monitoring, and so on. In the existing deep learning methods, the traditional convolutional neural network mainly focuses on local features, and the extracted image features are relatively large, which makes it easy to lose details in the image. To solve this problem, this paper proposes an infrared and visible image fusion algorithm, RSTSFusion, based on improved residual Swin Transformer and Sobel operators. Replace the convolutional neural network with an improved pure residual Swin Transformer network to realize the long-distance dependence between the global information and pixels of the image. The Efficient Channel Attention (ECA) mechanism added to the network can improve the network’s perception of important features and better capture global information. At the same time, a new fusion layer is designed, and the Sobel operator is added to enhance the extraction of image edge details, which can better preserve infrared features and clear visible details. The proposed method is compared qualitatively and quantitatively with four commonly used methods on TNO data sets. The experimental results show that our proposed method is superior to other methods in SF, MI, AG, SD, SCD, and other indicators and has obvious advantages in preserving global features and details.