Geometry-Injected Image-Based Point Cloud Semantic Segmentation

Image-based methods have replicated the success from 2-D domain to 3-D point cloud semantic segmentation. However, when we directly apply 2-D techniques to the projected pseudo-image, inherent differences between the point cloud and the image cause geometric distortion. This article analyzes the geometric distortion between the point cloud and the pseudo-image, including truncation, dislocation, and hole. To ensure geometric fidelity, we propose the Geometry-injected Image-based point cloud semantic segmentation Network (GINet). We design a cyclic convolution to optimize the convolution operation, dealing with truncation. For dislocation and hole, we propose dual geometric constraints, including local spatial attention and local affinity regularization, to incorporate the geometric information into semantic feature learning. Local spatial attention generates an attention map from the point coordinates to modulate the feature map before convolution. Local affinity regularization supervises the semantic similarity of pixels in the convolution kernel range. The GINet rectifies the geometric distortion with these mechanisms while taking advantage of the successful 2-D semantic segmentation methods. Quantitative and qualitative experiments on SemanticKITTI and SemanticPOSS demonstrate the effectiveness of GINet.