Augmenting reality for medicine, training, presence and telepresence

At least since Sutherland's 1968 head-mounted display, augmented reality systems have inspired (and frustrated) generations of developers, users, and enthusiasts. These inspirations have led to decades of effort that yielded major innovations in technologies for 3D capture, 3D displays, tracking, and real-time image generation. Some of these component technologies, such as head-mounted displays, have proven much more difficult to bring to widespread adoption than many of us expected. Others, such as real-time image generation, have become phenomenally successful, with major societal impacts extending far beyond the initial applications. In this talk, I will describe several developments in these areas, and illustrate the resulting systems that exploited them: 1) enhancing 3D scene acquisition by laser scanning, or with structured light, or with multiple acquisition cameras; 2) augmenting a physician's view of her patient with registered internal imagery; 3) augmenting a user's surroundings with projection onto multiple nearby surfaces; 4) augmenting a user's remote presence using a human-sized avatar that mimics appearance, pose, and gestures; 5) augmenting tabletop displays with multi-user autostereoscopic capabilities. The common goal of all these systems is to enrich users' immediate surroundings with computer-generated or -controlled enhancements, which can run the gamut from mere virtual imagery to full-fledged robotic androids. I will also speculate about the possible paths to progress in the coming years. The future is encouraging, as the decreasing cost of the necessary components lowers the barriers to entry and encourages ever-increasing participation in innovation, development and use. Coupled with the rapidly advancing technologies in sensors, cameras, displays, robotics, and networks, this should enable us to accelerate bringing the visions of augmenting reality to daily life.