A Light Gradient Residual Encoder-Decoder Network for Multimodal Image Fusion

Image fusion combines the complementary traits of source images into a single output, enhancing both human visual observation and machine vision perception. The existing fusion algorithms typically prioritize visual enhancement, often overlooking the real-time needs for critical surveillance applications. To address these real-time deployment needs, we present a compact fusion network for combining infrared and visible image representations, named Light-weight Fusion (LightFusion). This network employs incremental semantic integration and scene recognition accuracy constraints by incorporating three different bands of images (IR, RGB, and Grayscale) to fuse the data. Our approach includes a sparse semantic perception branch that captures critical semantic features, which are then integrated into the fusion network through a semantic injection module. This ensures that high-level vision tasks are adequately addressed. The scene fidelity path ensures that fusion features preserve all details required to reconstruct the original images. The importance and applicability of the proposed network are enhanced by employing an extra input in the form of a grayscale image, obtained by converting the RGB image for improved contrast, along with prominent target masks to enhance the visual quality of the fusion results. Our extensive analysis shows that the lightweight LightFusion network outperforms existing methods in both visual quality and semantic integrity, even under challenging conditions. The source code will be released at https://github.com/MI-HussainiLightFusion.