Augmenting moving planar surfaces interactively with video projection and a color camera

Traditional applications of augmented reality superimpose generated images onto the real world through goggles or monitors held between objects of interest and the user. To render the augmented surfaces interactive, we may exploit directly existing computer vision techniques. However, when using video projection to alter directly the appearance of surfaces, most vision-based algorithms fail. Even Wear Ur World [5], a recent and otherwise well-received interactive projector-camera system, relies on colored thimbles as markers. As notable exception, Tele-Graffiti [6] was designed for normal visible-light cameras without markers, but still considers the light emitted from the projector as unwanted interference, limiting its application.