Visible and Infrared Image Fusion Based on Modality Feature Enhancement for Localization in Low-Light Environments

In low-light environments, the imaging quality of single-modal red green blue (RGB) sensors severely deteriorates, leading to blurred textures and loss of effective information, which affects the accuracy of downstream tasks. Our goal is to provide robust feature extraction in low-light environments by fusing visible and infrared images. Infrared and visible fusion is an important and effective image enhancement technique, aiming to generate high-quality fused images with prominent targets and rich textures in challenging environments. Hence, we propose an unsupervised enhanced infrared and visible fusion method for low-light environments. The method first designs a module for single-modal image feature enhancement based on infrared and visible images for low-light conditions, initially improving the quality of the original infrared and visible images. Subsequently, a cross-modal feature enhancement module based on edge/texture information extraction and guidance is proposed to enhance the edge structures and texture details in the fused features. Specifically, the network utilizes the ResNet as the backbone for single-modal image feature extraction and enhancement, employing channel and spatial attention mechanisms to enhance single-modal image features. An infrared/visible image edge/texture information guidance module is added, leveraging the complementary edge/texture features provided by the two different modalities to guide the learning of the other modality image, thereby achieving the goal of cross-modal image enhancement in low-light environments. In the fusion stage, the DenseNet is employed as the unsupervised fusion network framework. Based on image information measurements, a patch-based loss function with regional weighting is designed, enabling the fused image to dynamically learn advantageous features from different regions of different modal images, achieving complementary feature enhancement of infrared and visible images. Through qualitative and quantitative analyses of three datasets, compared with eight other state of the art (SOTA) methods, the proposed method demonstrates balanced performance, achieving high-quality fusion of multimodal features in low-light environments.