Real-World Virtual Reality With Head-Motion Parallax

Most of the real-world virtual reality (VR) content available today is captured and rendered from a fixed vantage point. The visual–vestibular conflict arising from the lack of head-motion parallax degrades the feeling of presence in the virtual environment and has been shown to induce nausea and visual discomfort. We present an end-to-end framework for VR with head-motion parallax for real-world scenes. To capture both horizontally and vertically separated perspectives, we use a camera rig with two vertically stacked rings of outward-facing cameras. The data from the rig are processed offline and stored into a compact intermediate representation, which is used to render novel views for a head-mounted display, in accordance with the viewer’s head movements. We compare two promising intermediate representations—Stacked OmniStereo and Layered Depth Panoramas—and evaluate them in terms of objective image quality metrics and the occurrence of disocclusion holes in synthesized novel views.