Large-Scale 3D Point Cloud Processing for Mixed and Augmented Reality

The rapid development of 3D scanning technology combined with state-of-the-art mapping algorithms allows to capture 3D point clouds with high resolution and accuracy. The high amount of data collected with LiDAR, RGB-D cameras or generated through SfM approaches makes the direct use of the recorded data for realistic rendering and simulation problematic. Therefore, these point clouds have to be transformed into representations that fulfill the computational requirements for VR and AR setups. In this tutorial participants will be introduced to state-of-the-art methods in point cloud processing and surface reconstruction with open source software to learn the benefits for AR and VR applications by interleaved presentations, software demonstrations and software trials. The focus lies on 3D point cloud data structures (range images, octrees, k-d trees) and algorithms, and their implementation in C/C++. Surface reconstruction using Marching Cubes and other meshing methods will play another central role. Reference material for subtopics like 3D point cloud registration and SLAM, calibration, filtering, segmentation, meshing, and large scale surface reconstruction will be provided. Participants are invited to bring their Linux, MacOS or Windows laptops to gain hands-on experience on practical problems occuring when working with large scale 3D point clouds in VR and AR applications.