A Method for Constructing Synthetic Datasets for Weakly Textured Pipe Instance Segmentation

Instance segmentation is of great significance for the automatic picking of weakly textured pipes. However, it is hard to acquire synthetic datasets with a robust sim-to-real capability for training instance segmentation models due to the domain gap between the synthetic and real images. This paper proposes a method for generating photorealistic images of weakly textured pipes using BlenderProc2 and automatically constructing a synthetic dataset. The innovation of this approach lies in combining Physically-Based Rendering (PBR) with domain randomization techniques to reduce the sim-to-real gap. The presented results on instance segmentation tasks indicate that the Mask R-CNN model with PointRend module achieves a 9.919% improvement in Average Precision (AP) for segmentation and an almost 29% improvement for bounding boxes when trained on images synthesized using the proposed method, compared to training on realistic images combined with data augmentation. Thus, the effectiveness of the proposed method is verified in this study.