Applications of omnidirectional imaging: multi-body tracking and remote reality

Recently, S. Nayar (1997) introduced a parabolic imaging system that has a field of view of a full hemisphere or more. When used with a video camera the result is an omni-directional video stream that captures everything going around it. In the VAST Lab at Lehigh, we have been experimenting with these cameras, developing new variants, and developing omni-directional vision applications. We present an overview of omni-directional imaging and then two of our applications which we will be demonstrating. The first application is a frame-rate multi-body tracking system. The system uses an omni-directional imager and a standard PC to track multiple moving objects in all directions. The system is designed to provide perspective views of the most significant targets, either locally or over a network. The second application is something we call Remote Reality, which provides an immersive environment via omnidirectional imaging. It can use live or pre-recorded video from a remote location. While less interactive than traditional VR, remote reality has important advantages: there is no need for "model building" and the objects, textures and motions are not graphical approximations.