Mixed Reality Human Teleoperation

For many applications, remote guidance and telerobotics provide great advantages. For example, tele-ultrasound can bring much-needed expert healthcare to isolated communities. However, existing tele-guidance methods have serious limitations. A new concept called human teleoperation leverages mixed reality, haptics, and high-speed communication to provide tele-guidance that is more tightly coupled than existing methods yet more accessible than telerobotics. This paper provides an overview of the human teleoperation concept and its application to tele-ultrasound. The concept and its impact are discussed, the graphics, communications, controls, and haptics subsystems are explained, and results are presented that show the system's efficacy. These include tests of the communication architecture, of human performance in tracking mixed reality signals, and of human teleoperation in a limited clinical use-case. The results show good potential for teleultrasound, as well as possible other applications of human teleoperation including remote maintenance, inspection, and training.