Multimodal Fusion for Brain Medical Image Segmentation using MMSegNet

Multimodal medical image fusion plays a crucial role in integrating information from various imaging modalities, it involves combining data from different models to obtain more and more detailed information, resulting in better diagnostic accuracy compared to using a single model. The need for innovation in this field stems from the importance of semantic medical information (MS-Info) presented in many forms. Effective capture and use of MS-Info leads to better disease identification and diagnosis. In this work, we present a new model for CNN that establishes the relationship between MSInfo and image features to guide inference. The model has two branches; Spatial frequency (SF) (for MR) and grayscale HSV (GV) (PET/SPECT/CT) images used to fuse images by keeping core features to contribute to more detailed disease characterization, ultimately improving patient care and treatment planning. The success of our proposed model has the potential to revolutionize many fields of medical imaging fusion and improve the understanding and diagnosis of various brain diseases.