A hybrid image-based and model-based telepresence system using two-pass video projection onto a 3D scene model

A telepresence system is presented that has the advantage of both model-based and image-based approaches, namely, free viewpoint control and real-time color update with live video. A remote place is presented as a virtual environment by using live video projection captured by a head-worn camera onto the static 3D geometry. The observer can then observe the remote place in cooperation with the remote camera man, and give him a set of 3D instructions by a mouse.