Real-time Upper Body Reconstruction and Streaming for Mixed Reality Applications

In view of the challenges of real-time 3D reconstruction and transmission, the research on tele-immersion systems has been quite intense. We present an end-to-end, realtime 3D reconstruction system of the human body's upper part in mixed reality applications, implemented with the use of a single depth camera on the capture side, whereas no special setup is required. Our system captures the scene, extracts the user's point cloud and by quantizing it, achieves real-time mesh generation and streaming. This system, together with the appropriate virtual (VR) or augmented (AR) reality equipment, creates a sense of a more direct, face-to-face communication, as if both users were in the same environment.