Mobile Face Capture for Virtual Face Videos

Minimally obtrusive face capture is essential for applications in tele-conferencing, collaborative work, and mobile applications. Acquiring and coding video for such applications is a challenging problem, particularly for a mobile user. A pilot head-mounted display system (HMD) is shown that captures two side views of the face and generates in real-time a quality frontal video of the wearer. The face is captured with little obstruction of the users field of view. The frontal views are generated by warping and blending the side views after a calibration step. In tests of a bench prototype, the generated virtual videos compared well with real video based on both objective and qualitative criteria. With success of the pilot work, we are continuing development of the mobile HMD system.