Robust 3D Semantic Segmentation With Incomplete Point Clouds Based on Sequential Frame Sampling

This paper proposes a method for learning 3D semantic segmentation robust to incomplete point clouds. Our method first generates pseud-incomplete point clouds from original 3D point clouds by sequential frame sampling that creates multiple subsets considering the continuity of an RGB-D sequence for reproducing incomplete areas in the point clouds. It then simultaneously learns completion networks and semantic segmentation networks with the pseud-incomplete point clouds. We evaluate our method on the 3D semantic segmentation task. Experimental results on ScanNet v2, an indoor environment, show that our method improves mIoU by 0.4 points for the original point clouds and 6.3 points for the incomplete point clouds compared with a conventional method. Experimental results on WorkPlace Dataset, an outdoor environment, show that our method improves mIoU by 6.5 points for the original point clouds and 11.1 points for the incomplete point clouds compared with the conventional method. These results improve the safety and operability of environmental awareness in applications such as robotics.