SPMRFusion: SPECT and MRI Image Registration and Fusion Based on TransGAN and Self-Attention Mechanism

Single-modality medical imaging is constrained by sensors, limiting simultaneous capture of functional-metabolic and anatomical details, thus compromising diagnostic accuracy. Multimodal fusion addresses this problem by integrating complementary modality information to enhance surgical guidance and clinical decision-making. However, despite advancements in pre-aligned multimodal fusion, SPECT-MRI cross-modality fusion remains hindered by spatial mismatches (e.g., tissue deformation, registration error) and geometric distortions stemming from their disparate imaging mechanisms. To address this shortcoming and realize precise registration and fusion, we put forward a novel SPMRFusion network framework for unsupervised multimodal medical image registration and fusion, which is based on self-attention mechanism. In this methodology, we incorporate a combined GAN and Swin Transformer framework to promote image registration, thereby mitigating spatial shift and ghosting. Moreover, the aligned SPECT and MRI information undergoes self- attention fusion to generate the network's output images. Both qualitative and quantitative experimental outcomes showcase the method's excellent performance in the fusion of medical SPECT-MRI images.