Registration for stereo vision-based augmented reality based on extendible tracking of markers and natural features

This paper describes a method to extend the registration range of a vision-based augmented reality (AR) system. We propose to use natural feature points contained in images captured by a pair of stereo cameras in conjunction with pre-defined fixed fiducial markers. The system also incorporates an inertial sensor to achieve a robust registration method which can handle user's fast head rotation and movement. The system first uses pre-defined fiducial markers to estimate a projection matrix between real and virtual coordinate systems. At the same time, the system picks up and tracks a set of natural feature points from the initial image. As a user moves around in an AR environment, the initial markers fall out from the camera frame and natural features are then used to recover the projection matrix. Experiments for evaluating the feasibility of the method are carried out and show the potential benefits of the method.