Reducing domain shift in synthetic data augmentation for semantic segmentation of 3D point clouds

The use of deep learning in semantic segmentation of point clouds enables a drastic improvement of segmentation precision. However, available datasets are restrained to a few applications with limited applicability to other fields. Using synthetic and real data can alleviate the burden of creating a dedicated dataset at the cost of domain-shift that is mostly addressed during training, while treating the problem directly on the data has been less explored. Towards this goal, two methods to alleviate domain shift are proposed, firstly by enhanced generation and sampling of synthetic data and secondly by leveraging color information of unlabeled point clouds to color synthetic, uncoloured data. Obtained results confirm their usefulness in improving semantic segmentation result (+3.43 into mIoU for a network trained on S3DIS zone 1). More importantly, the devised coloring method shows the ability of a point-based network to link color information with recurrent geometric features. Finally, the presented methods are able to bridge the domain-shift gap even in cases where inclusion of raw synthetic data during training impedes learning.