Immersive Human-Robot Interaction for Dexterous Manipulation in Minimally Invasive Procedures

Interaction between humans and robots is inevitable in robotic surgeries. The purpose of this work is to enhance human-robot interaction for computer-assisted minimally invasive surgeries through immersive technology. Our developed system includes a robotic manipulator designed to perform minimally invasive surgeries and a mixed reality-based interface that facilitates immersive interaction between a human operator and a collaborative robot. The system provides visual and dexterous augmentation through the immersive projection of the endoscopic view in proximity to the surgical site and a virtual remote-center-of-motion (RCM) intuitively specified by the users. Experimental observation demonstrated sub-millimeter and sub-degree accuracy between the spatial representation of the physical robot and its virtual model. Safety design and robot intelligence to enhance the interaction between humans and robots via MR technology will further be developed to prepare the technology for actual clinical use in the long run.