Altering reality through interactive image and video manipulation

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe an image-based approach to augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment's objects. The only requirement is the ability to track across frames at least four feature points that are specified by the user during system initialization and whose world coordinates are unknown. The resulting system allows a user to "augment" a real 3D environment with computer-generated objects by interactively manipulating one or more images of that environment.