Application of Natural User Interface Devices for Touch-Free Control of Radiological Images During Surgery

Natural User Interface (NUI) systems can enable the scrubbed clinician to assume direct control of medical image interaction while maintaining sterility in the Operating Room. Surgeons and radiologists trialed a touch-free image control system based on the Leap Motion and Microsoft Kinect v2 controllers. Feedback was reported on the perceived utility and usability of both devices. The speed and accuracy of the two controllers was measured. Results showed marginal to average acceptability of both controllers. Surgeons and Interventional Radiologists found Microsoft Kinect to have better utility and to be potentially useful for the majority (54%) of them. The accuracy of the Leap Motion sensor was superior and comparable with that of a computer mouse. A link was established between the system usability and the perception of utility with better usability translating into better utility. Advantages and limitations of each device are highlighted. Design improvements and deployment considerations are discussed.