Endomersion: An Immersive Remote Guidance and Feedback System for Robot-Assisted Minimally Invasive Surgery

The current practice of surgical telementoring involves remote experts providing guidance via telephone or video conferences. Especially in minimally invasive surgery (MIS), this greatly limits their insight into the procedure and inhibits effective deictic referencing. To address this, we present Endomersion, an immersive remote guidance and feedback system that allows verbal and non-verbal communication through audio chat, pointing and telestration, enables remote control of the endoscope camera, and offers an immersive 3D workspace supporting remote expert’s understanding of the on-site operation. The system was developed with feedback from surgeons and tested in actual operating room settings. In this demonstration, users take over the role of the remote expert, communicating with the on-site surgeon and remotely controlling the endoscopic camera attached to a robot arm.