Plane-dominant object reconstruction for robotic spatial augmented reality

We present a simple reconstruction algorithm of a plane-dominant 3D environment for the robotic spatial augmented reality (RSAR). In spatial augmented reality, a projector renders virtual objects onto 3D objects in the real space. To watch the augmented virtual objects from a viewpoint without distortions, the final image should be pre-distorted based on the geometry information of the 3D objects in the real world. In our RSAR setting, we assume that a robot is equipped with the devices such as a projector, low-cost depth cameras, and the sensor of capturing 3D position of the viewpoint. The robot captures the 3D environment with as a point cloud and reconstructs the geometry of 3D objects in the real world with a set of planes. In order that the viewer can see the distortion-free virtual objects, we compute the pre-warped images of virtual objects for a projector. Finally, we provide an efficient algorithm using GPU shaders to compute the pre-warped images for a projector. In experiments, we provide some results of preliminary simulations for our RSAR scenario.