A Collaborative Fusion and Registration Framework for Multimodal Image Fusion

With the advancement of Internet of Things (IoT) technology, multimodal data fusion plays a crucial role in applications, such as intelligent monitoring and environmental sensing. In the task of fusing infrared and visible images, IoT can integrate various sensors and cameras, providing comprehensive environmental information across different times and perspectives. However, existing fusion methods often result in significant artifacts in the fused images when misaligned inputs are involved, adversely degrade the effective integration of information. Moreover, the substantial differences in multimodal images impose higher precision requirements on image registration tasks. Although some current methods have achieved results in specific scenarios, they typically treat image registration and fusion as independent tasks, failing to achieve synergistic optimization. Therefore, this study considers image fusion and registration as a joint task and employs a joint training strategy. Specifically, a coarse-to-fine registration approach is implemented, supported by a specially designed multiscale progressive sequence feature module to enhance the accuracy of the registration network. Additionally, a multimodal hybrid-aware fusion module is incorporated into the fusion network to achieve adaptive feature fusion, ensuring that more textural information is preserved in the fused images. The fused image serves as feedback to further improve registration accuracy, which in turn enhances the fusion results. Comparative experiments with a wide range of state-of-the-art registration and fusion methods demonstrate the superiority of the proposed approach.