Patient 3D Data Visualisation with AR-based Interactive Technology for Brain MRI

Effective 3D patient data visualization for brain MRI is required in surgical planning and in-situ guidance during brain tumor resection. Traditional 2D displays do not possess depth perception and are poor in understanding complex anatomical structures being displayed in 3D. In this work, an interactive visualization system based on augmented reality is presented, which provides an intuitive and truly immersive way to visualize and interact with 3D brain MRI data as preoperative planning and in-situ guidance by a surgeon during tumor resection procedures of the brain. In this work, we propose a system where AR Devices can render patient-specific 3D brain models, reconstructed from MRI data, in a spatially registered way to the surgical field. These 3D virtual models can be interacted by the surgeon, through the natural use of controls, to study the spatial relationship of critical structures such as tumors, ventricles, and functional brain regions. Advanced volume rendering and multi-modal data fusions are used in creating photorealistic 3D visualizations with high anatomical detail.