3D Instance Embedding Learning With a Structure-Aware Loss Function for Point Cloud Segmentation

This letter presents a framework for 3D instance segmentation on point clouds. A 3D convolutional neural network is used as the backbone to generate semantic predictions and instance embeddings simultaneously. In addition to the embedding information, point clouds also provide 3D geometric information which reflects the relation between points. Considering both types of information, the structure-aware loss function is proposed to achieve discriminative embeddings for each 3D instance. To eliminate the quantization error caused by 3D voxel, the attention-based $k$-nearest neighbor (kNN) is proposed. Different from the average strategy, it learns different weights for different neighbors to aggregate and update the instance embeddings. Our network can be trained in an end-to-end style. Experiments show that our approach achieves state-of-the-art performance on two challenging datasets for instance segmentation.