An Optical See-Through Augmented Reality System with Gesture-Based Interaction

We present an optical see-through augmented reality system. The system features optical see-through calibration, object tracking, virtual scene rendering and gesture interaction. Compared with video see-through augmented reality system, the optical see-through system requires more accurate alignment between the virtual and the real scene. Our system calibration is done using natural feature based calibration method based on SPAAM algorithm. The tracking method is based on RGB-D camera and the gesture interaction is implemented with custom gestures. Finally, we design a test scene and the system achieves the mixture of virtual and real reality in the test scene and meets the accuracy and real-time requirement.