PolishNet-2d and PolishNet-3d: Deep Learning-Based Workpiece Recognition

In recent years, there have been many deep learning research projects based on two-dimensional object detection and three-dimensional point cloud recognition. However, relatively few of these combine the two, and the number of projects based on a three-dimensional workpiece recognition method for the industrial field is even fewer. In this paper, to perform the recognition task for polishing workpieces in manufacturing, we use RGB-D images as input, and propose our designed PolishNet-2d for two-dimensional workpiece detection and our designed PolishNet-3d for three-dimensional workpiece recognition. The two deep neural networks are employed together in series to first detect and then recognize polishing workpieces in an industrial environment. For this paper, a large number of experiments have been carried out on deep learning datasets of polishing workpieces. These datasets were created by us to contain diverse combinations of real and simulated workpieces in real and simulated industrial environments. The contributions of this paper are as follows: (1) A rotation parameter learning network is proposed and a two-dimensional workpiece detection neural network, named PolishNet-2d, which was constructed by integrating our designed algorithms with the backbone networks ResNet101 and region proposal network (RPN), is introduced; (2) A hierarchical feature extraction network is proposed and a three-dimensional workpiece recognition neural network, named PolishNet-3d, which was constructed by integrating our designed algorithms with the backbone network PointNet, is introduced; (3) PolishNet-2d and PolishNet-3d are employed in series, with the detection output of PolishNet-2d being used as the input for PolishNet-3d for its recognition tasks: the workpiece regions are detected in the RGB image; the workpiece point cloud is segmented in the corresponding regions in the depth image, and lastly the segmented point cloud is placed into PolishNet-3d to identify the workpiece; (4) For the experiments in this paper, datasets containing rich and diverse data types of polishing workpieces for industrial fields have been constructed; (5) Numerous experimental results on polishing workpiece datasets show that the conjunction of PolishNet-2d and PolishNet-3d can achieve exemplary recognition results on polishing workpiece datasets.