3D Visualization System of Breast Magnetic Resonance Images Based on Deep Learning and Volume Rendering

Compared to natural images, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) exhibits lower spatial resolution, reduced contrast sensitivity, and limited detail visibility, making it time-consuming and labor-intensive for clinicians to delineate lesion areas. Additionally, patients often struggle to understand the condition of the lesions intuitively. To address these challenges, a three-dimensional (3D) visualization and interactive system for breast DCE-MRI data is proposed and implemented, which integrates deep learning-based segmentation with volume rendering techniques. The multimodal feature fusion network (MFF-Vnet) based on the Vnet architecture employs a multi-branch design tailored to process multimodal images. Specifically, the MFF-Vnet model automatically segments breast lesions, providing accurate delineation of lesion areas. Then, an improved ray-casting algorithm is utilized for volume rendering to visualize the 3D structure of the breast and associated lesions with enhanced speed and rendering quality. Finally, an interactive visualization system is developed based on a client/server (C/S) three-tier architecture, enabling functions such as data loading, slice visualization, lesion segmentation, 3D rendering, and interface interaction. Experimental results demonstrate that the improved ray-casting algorithm accelerates the rendering process while maintaining visualization quality. The MFF-Vnet model demonstrated promising segmentation performance on a breast DCE-MRI dataset containing expert-annotated lesions, achieving a Dice Similarity Coefficient (DSC) of up to 83.77%. Through interactive operations, the system facilitates multi-scale and multi-perspective observation and analysis of lesion characteristics, including shape, size, and location, thus assisting clinicians in diagnosis and treatment planning, while improving patient understanding of their condition.