An Integrated Model for Generating 3D Point Clouds and Segmentation from 2D Images

This study presents a cost-effective approach to constructing simulation systems by integrating 2D visual processing with neural network technology for simultaneous 3D point cloud generation and part segmentation, thereby reducing time and costs. We propose multiple task-specific models using 2D RGB images, incorporating attention mechanisms, feature encoding, decoding layers, and innovative loss functions. The optimal models are merged to form an integrated system, evaluated using public and self-built datasets. Our experiments demonstrate that the integrated model effectively reduces training time and Floating Point Operations (FLOP) values while maintaining high accuracy in Critical Hole Distance (CHD), Earth Mover’s Distance (EMD), and Intersection over Union (IOU) metrics. This advancement enhances applications in military training, aerospace technology, and disaster response simulation, facilitating faster 3D object construction.