Optical see-through calibration with vision-based trackers: propagation of projection matrices

Recently, M. Tuceryan and N. Navab (2000) introduced a method for calibrating an optical see-through system based on the alignment of a set of 2D markers on the display with a single point in the scene, while not restricting the user's head movements (the single point active alignment method or SPAAM). This method is applicable with any tracking system, provided that it gives the pose of the sensor attached to the see-through display. When cameras are used for tracking, one can avoid the computationally intensive and potentially unstable pose estimation process. A vision-based tracker usually consists of a camera attached to the optical see-through display, which observes a set of known features in the scene. From the observed locations of these features, the pose of the camera can be computed. Most pose computation methods are very involved and can be unstable at times. The authors propose to keep the projection matrix for the tracker camera without decomposing it into intrinsic and extrinsic parameters and use it within the SPAAM method directly. The propagation of the projection matrices from the tracker camera to the virtual camera, representing the eye and the optical see-through display combination as a pinhole camera model, allows us to skip the most time consuming and potentially unstable step of registration, namely, estimating the pose of the tracker camera.