Learning Physical Activities in Immersive Virtual Environments

This paper describes a framework for constructing a three-dimensional immersive environment that can be used for training physical activities. In the proposed system, an immersive environment is constructed through realistic reconstruction of a scene by applying a stereo algorithm on a set of images that is captured from multiple viewpoints. Specifically, twelve camera clusters that consist of four camera quadruples are used to capture full body human motions, where each camera cluster is processed by a pc independently and synchronously so that they can compute 3d information of the scene in real-time fashion. This paper discusses in detail system architectures that enable synchronous operations across multiple computers while achieving parallel computations within each multi-processor system. A set of experiments are performed to learn taichi lessons in this environment where students are instructed to follow pre-recorded teacher's movements while observing both their own motions and the teacher in real-time from arbitrary viewpoints.