Point Cloud Instance Segmentation of Indoor Scenes Using Learned Pairwise Patch Relations

Indoor scene understanding is an active research topic that has recently gained increasing attention in both computer graphics and computer vision community. This paper presents a novel automatic point cloud instance abstraction and segmentation of indoor scenes using learned pairwise planar patch relations. Planar patches have sufficient representative power to abstract the semantics in mostly man-made 3D indoor scenes. To exploit the planar patch relationships, a new indoor point clouds dataset is automatically derived from the public dataset ScanNet, which can be adopted for training a pairwise patch relation classifier. Given the pre-processed indoor scene point clouds, a set of planar patches are generated to provide robust and compact representation of indoor objects and parts. For each extracted planar patch, a feature descriptor in view of its geometric properties and appearance is calculated. Owing to the proposed feature descriptors, the relations of pairwise planar patch are measured and can be fed into the pre-trained patch relation classifier. To yield the instance segmentation of indoor scenes, a robust graph-based algorithm is adopted to group these planar patches which incorporate the learned patch relations. Experimental results demonstrate that our proposed approach can produce instance semantic segmentation of various cluttered indoor scenes robustly and efficiently.