Deep Non-Linear Embedding Deformation Network for Cross-Modal Brain MRI Synthesis

Multimodal MRI (e.g. T1, T2, and Flair) can provide rich anatomical and functional information, thereby facilitating clinical diagnosis and treatment. However, multimodal MRI takes a long scan time, easily leading to artifacts or corruption in certain modalities. Therefore, it is of great value to synthesize a new MRI modality from a complete MRI modality to obtain complementary information for clinical diagnosis. Existing GAN-based approaches treat cross-modal MRI synthesis as an end-to-end learning process without explicit consideration of the inherent correlations between different modalities, leading to inaccurate anatomical and lesion structure in the synthesized modality. In this paper, we propose a deep non-linear embedding deformation network (NEDNet) for cross-modal brain MRI synthesis. NEDNet represents each modality as a non-linear embedding based w.r.t. its own atlas, and learns a deformation feature that is assumed to be the same across modalities. The modality-specific atlas and multi-modal shared deformation are jointly used for generating the new MRI modality. Experiments show that our approach can obtain better cross-modality synthesis results than several baseline methods.