Development of the Varioscope AR. A see-through HMD for computer-aided surgery

In computer-aided surgery (CAS), an undesired side-effect of the necessity of handling sophisticated equipment in the operating room is the fact that the surgeon's attention is drawn from the operating field, since surgical progress is partially monitored on the computer's screen. Augmented reality (AR), the overlay of computer-generated graphics over a real-world scene, provides a possibility to solve this problem. The technical problems associated with this approach, such as viewing of the scenery within a common focal range on the head-mounted display (HMD) or latency in the display on the HMD, have, however, kept AR from widespread usage in CAS. The concept of the Varioscope AR, a lightweight head-mounted operating microscope used as a HMD, is introduced. The registration of the patient to the pre-operative image data, as well as pre-operative planning, take place on VISIT, a surgical navigation system developed at our hospital. Tracking of the HMD and stereoscopic visualisation take place on a separate POSIX.4-compliant real-time operating system running on PC hardware. We were able to overcome the technical problems described above; our work resulted in an AR visualisation system with an update rate of 6 Hz and a latency below 130 ms. It integrates seamlessly into a surgical navigation system and provides a common focus for both virtual and real-world objects. First evaluations of the photogrammetric 2D/3D registration have resulted in a match of 1.7 pixels on the HMD display. The Varioscope AR with its real-time visualisation unit is a major step towards the introduction of AR into clinical routine.