Feedback from video for virtual reality navigation

Important preconditions for wide acceptance of virtual reality systems include their comfort, ease and naturalness to use. Most existing trackers suffer from discomfort related issues. For example, body-based trackers (such as hand controllers, joysticks or helmet attachments) restrict spontaneity and naturalness of motion, whereas ground-based devices (e.g., hand controllers) limit the workspace by literally binding an operator to the ground. Controls have similar problems. This paper describes using real-time video with registered depth information (from a commercially available camera) for virtual reality navigation. A camera-based setup can replace cumbersome trackers. The method includes selective depth processing for increased speed, and a robust skin-color segmentation for handling illumination variations.