Easy Extrinsic Calibration of VR System and Multi-camera Based Marker-Less Motion Capture System

This paper proposes a novel easy extrinsic calibration algorithm for an off-the-shelf VR system and a multi-camera based marker-less motion capture system. To realize interactions between 3D user motions and virtual objects reconstructed from multi-view videos in a common 3D space, the extrinsic calibration of the VR system and the multiple cameras must be conducted beforehand. This calibration, which involves estimating the pose and position of each coordinate system, is a key technology for handling 3D information in a system with various type of input sources. In general, extrinsic calibration is carried out by identifying and utilizing some common 3D points. However, since most of off-the-shelf VR systems do not include any imaging device, it is difficult to apply conventional automatic calibration approaches as there are no common points shared with the cameras. Against this problem, this paper introduces an easy calibration algorithm by generating corresponding points from the trajectories of the user's motion with VR devices and 3D human pose reconstructed from multi-view videos. Our study provides the following two contributions; (1) our method does not need to introduce additional devices, such as a chessboard and (2) our method does not need manual processes as the extrinsic parameters are automatically estimated. We demonstrate the performance of the proposed method in a practical scenario.