High-Resolution Stereoscopic Visualization of Pediatric Echocardiography Data on Microsoft HoloLens 2

Three-dimensional ultrasound offers volumetric images and detailed anatomical data for medical diagnosis and treatment planning. It is a key tool in the medical field to obtain a comprehensive view of the body. Ordinary two-dimensional displays do not provide depth perception and are not suitable for representing volumetric data, necessitating the use of more sophisticated visualization methods. Virtual and augmented reality (AR) displays can be used to improve the visualization of medical images, allowing for more natural interaction with the environment. This study proposes custom software developed using the Unity3D platform to render high-resolution 3D echocardiography (3DE) on the Microsoft HoloLens 2, providing an immersive AR experience for medical professionals. This research focuses on three-dimensional echocardiography in children and uses a phantom heart model to mimic a pulsating heart. The volume rendering algorithm utilizes the ray-marching technique, enabling direct volume rendering of high-quality volumetric models. To maintain a satisfactory frame rate, a Holographic Remoting approach is employed to reduce latency and enhance network transmission speed, utilizing the resources of a personal computer (PC). The custom software developed offers an intuitive and interactive user interface that allows medical professionals to manipulate and explore 3DE images effectively. The interaction includes the ability to slice, modify the intensity range, and alter the voxel density. The experimental evaluations demonstrated that it is possible to produce high-quality real-time display with HoloLens 2 and a PC-based remote rendering system, allowing intuitive control and exploration of 3DE. Overall, this research highlights the potential of AR rendering offered through Microsoft HoloLens 2 to advance pediatric 3DE rendering for medical professionals to enhance their decision-making and understanding of medical datasets.