A Real-time 3D Perception Method with Multi-resolution Voxel Space

3D perception is the key technology of intelligent equipment to understand the world. We propose an online 3D perception method from RGBD sequences which consists of 3D instance segmentation, camera pose optimization, 3D reconstruction, and refinement modules. Firstly, we perform 3D instance segmentation by mapping and fusing 2D instance segmentation in 3D space. Secondly, we obtain accurate camera poses by adopting a frame-to-frame optimization method and local-to-global strategy. Thirdly, we reconstruct 3D scenes by a multi-resolution implicit structure based on voxels with semantic information. Finally, we render refined implicit surfaces by an improved rendering method. Experiments show that our method has better 3D perception results in a variety of complex environments compared with similar methods. Our code is available: https://github.com/3D-Vision-Lab-of-JUST/Real-time-3D-Perception.