Real-time Interactions and Synchronization of Voxel-based Collaborative Virtual Environments

Collaborative virtual environments (C-VE) facilitate team-oriented training on virtual reality-based surgical simulators. Many C-VEs replicate the VE on each user's machine to allow for real-time interactions. However, this solution does not work well when modifying voxel-based C-VEs because large and frequent volumetric updates make it difficult to synchronize the C-VE. This paper describes a hybrid depth-buffered image (DBI) and geometry-based rendering method created to simulate visual interactions between local virtual bone cutting tools and remotely maintained volumetric bone material for a craniotomy simulator. For real-time interactions, users only store a DBI of the volumetric C-VE and composite it with rendered images of surgical tools. Additionally, we describe methods to combat network bandwidth/latency to remotely simulate haptic and bone drilling interactions between users' tools and the volumetric VE. For haptic feedback, a multi-rate solution (Cavusoglu and Tendick, 2000) allows users to construct a local approximation of the volumetric C-VE to compute new forces. Only 2D DBI updates are required to synchronize different users when the bone changes due to drilling. Our approach provides an improved performance over a replicated VE that uses 3D model-based updates