TBFusion: Multimodal Medical Image Fusion via Tri-Branch Feature Reconstruction

Due to the complexity and diversity of disease, it is challenging to diagnose and locate lesions through single-modality medical imaging. Multimodal medical image fusion technology can effectively extract and integrate complementary information from multiple sources into a more understandable fused image. To address the issues of inadequate and inaccurate extraction of shared information and unique features across modalities, we propose a novel method to fuse multimodal medical images via tri-branch feature reconstruction, termed TBFusion. Paired source images are jointly fed into cross-modal feature constraint module (CFCM), which utilizes channel feature constraint to extract shallow common information. Simultaneously, the multimodal images are separately fed into multi-directional detail preservation module (MDPM) and adaptive hybrid attention module (AHAM) to retain the texture information of the structural images and focus on the metabolic information of the functional images. Compared to existing methods on feature decomposition, the tri-branch structure can automatically extract common and unique features from paired source images, reducing information redundancy. Additionally, multi-feature fusion module (MFM) is used to integrate these complementary features, resulting in high-quality fused images. Both qualitative and quantitative experiments demonstrate that our method outperforms the state-of-the-art.