M3CI-Net: Multi-Modal MRI-Based Characteristics Inspired Network for IDH Genotyping

Isocitrate dehydrogenase (IDH) is a key molecular feature for gliomas, and the prediction of IDH is also an important task for computer-aided diagnosis using magnetic resonance imaging (MRI). To address this changllenge, we introduce a multi-modal MRI-based characteristics inspired network for IDH Genotyping (M
³
CI-Net), which pay more attention to the different characteristics information of different MRI modalities T1, T2, T1ce, Flair. In M
³
CI-Net, a pre-fusion module with multi-channel attention mechanism is used to fuse T1ce and Flair modalities and capture as much as possible luminance and contrast information, and the edge information is obtained from T2 modality by using edge detection module. Finally, the feature information between modalities are fused and input into a CNN-Transformer based encoder structure to extract shared spatial and global information from multi-modal MRI, and the information of multiple scales frome encoder are input into the linear layer for IDH genotype classification after pooling, meanwhile, the CNN based decoder with skip-connection for glioma segmentation works for assisting IDH genotyping. Then, we proposed images’ pre-fusion loss, segmentation loss, IDH genotyping loss, and use uncertainty weight training method to balance the weights of these loss. we evaluate our proposed method on Brats2020, and achieve an acceracy of 0.88, an AUC of 0.94, a specificity of 0.92, a sensitivity of 0.84 in IDH genotyping, which is superior to the state-of-the-art methods.