USTNet: A U-Net Swin Transformer Network for Aerial Visible-to-Infrared Image Translation

Aerial visible-to-infrared image translation technology aims to generate infrared images from visible inputs, effectively expanding the acquisition of infrared images in complex aerial remote sensing scenarios. Although existing convolutional neural network (CNN)-based approaches demonstrate proficiency in capturing local spatial details, they often fail to model global scene context, which is crucial for maintaining structural consistency in complex aerial scenes. To address this challenge, a hybrid architecture network is proposed to model long-range dependencies while preserving fine-grained local details via an encoder–decoder framework. We further introduce a dynamic parallel window-based attention (DPWA) mechanism, which dynamically parallelizes window-based and shifted window-based multihead self-attention (W-MSA) in separate streams across transformer blocks to enhance global context modeling. Additionally, a masked image pretraining framework with wavelet transform loss is designed to guide multiscale feature alignment and high-frequency detail reconstruction, effectively addressing the texture discrepancy between visible and infrared modalities. Extensive experiments conducted on the upgraded aerial visible-to-infrared image translation tasks (AVIID) and DroneVehicle benchmark datasets demonstrate that our method significantly outperforms current state-of-the-art (SOTA) approaches in terms of both visual quality and perceptual metrics. The code of U-Net Swin Transformer network (USTNet) and the upgraded AVIID dataset are available at https://github.com/silver-hzh/USTNet