3D Brain Tumor Segmentation using U-Net: Integrating Deep Learning and Mixed Reality for Medical Imaging and Education

Precise diagnosis and treatment of gliomas, as well as other types of intracranial abnormalities, depend upon accurate segmentation of brain tumors. Recent advances in deep learning have profoundly influenced automation of this task, but a clear consensus on relative performance of 2D versus 3D CNN architectures, both in terms of segmentation accuracy and computational efficiency, remains outstanding, particularly in regard to clinical deployment. This study offers a comparative analysis of a 2D U-Net model and a 3D U-Net model. The 2D U-Net achieved a mean Dice Similarity Coefficient of 51.53%, coupled with a 94% accuracy, using an average training time of over five minutes per epoch on a single NVIDIA T4 GPU. For comparison, a 3D U-Net performed better, achieving a 71.66% DSC, 98.62% accuracy, and a mean Hausdorff distance (HD95) of 7.37-11.49 across various tumor sub-classes, at a training time of around a minute per epoch on a single NVIDIA A100 GPU. While a 3D model provides better volumetric context along with segmentation quality, it has much higher computational needs, with hardware properties having a profound effect on training efficiency. Furthermore, a Mixed Reality (MR) prototype is presented, integrating the predicted tumor masks into an interactive 3D environment to support clinician engagement with segmented MRI data. The integration of deep learning–based segmentation into MR tools highlights the prospect of more real-time, intuitive analysis in diagnostic and neurosurgical planning workflows. The results form a foundation for further work towards more advanced architectures, e.g., variants of Attention U-Net, and optimization of MR-based clinical visualization tools.