A stereo vision-based augmented reality system with marker and natural feature tracking

Proposes a method to extend the registration range of a vision-based augmented reality system. We propose to use natural feature points contained in images captured by a pair of stereo cameras in conjunction with pre-defined fixed fiducial markers. The system also incorporates an inertial sensor to achieve a robust registration method which can handle user's fast head rotation and movement. The system first uses pre-defined fiducial markers to estimate a projection matrix between real and virtual coordinate systems. At the same time, the system picks up and tracks a set of natural feature points from the initial image. As a user moves around in an AR environment, the initial markers fall out from the camera frame and natural features are then used to recover the projection matrix. Experiments evaluating the feasibility, of the method are carried out and show the potential benefits of the method.