Hybrid Feature Tracking and User Interaction for Markerless Augmented Reality

We describe a novel markerless camera tracking approach and user interaction methodology for augmented reality (AR) on unprepared tabletop environments. We propose a real-time system architecture that combines two types of feature tracking methods. Distinctive image features of the scene are detected and tracked frame- to-frame by computing optical flow. In order to achieve real-time performance, multiple operations are processed in a multi-threaded manner for capturing a video frame, tracking features using optical flow, detecting distinctive invariant features, and rendering an output frame. We also introduce a user interaction for establishing a global coordinate system and for locating virtual objects in the AR environment. A user's bare hand is used for the user interface by estimating a camera pose relative to the user's outstretched hand. We evaluate the speed and accuracy of our hybrid feature tracking approach, and demonstrate a proof-of-concept application for enabling AR in unprepared tabletop environments using hands for interaction.