RGB-D Camera Network Calibration and Streaming for 3D Telepresence in Large Environment

With the advance of multimedia technology, traditional remote communication applications (e.g. video chat) may not satisfy the increasing demand for effective interactivity due to the lack of realistic experience of being physically present. Telepresence is a new form of communication that aims to strengthen the connection between people by creating more immersive and interactive environments. However existing telepresence systems often suffer from limited viewing space or failing to render realistic 3D contents on the display. In this paper, we present a novel system that can improve the interactive experience among remote users with dynamically generated virtual 3D environments. To support wide view rendering, an automated calibration system is developed for RGB-D camera network that is scalable to capture and reconstruct large environments. Our proposed client-server architecture is capable of transmitting and processing large volume of RGB-D data across the the network in real-time. Virtual 3D views are automatically synthesized by seamlessly fusing multiple video streams acquired by different cameras. Our experimental results demonstrate better visual effects that can be applied to existing telepresence systems for more motivating and engaging remote applications.