Multi-Source Heterogeneous Factor Target Feature Segmentation and Dual-Arm Servo Harvesting Method for Grape Dual-Arm Robot on Horizontal Trellises

In grape trellis, a variety of obstacles can disrupt robots' high-speed movements along horizontal trellises. This paper introduces a method to identify and avoid these obstacles, using an improved Deeplab V3+model for semantic segmentation. The approach involves replacing the MobileN et V2 network with a lighter version, using a ReL U 6 activation function to maintain accuracy, and employing hybrid dilation convolution to reduce feature impact. The dataset includes common obstacles like branches, leaves, stalks, and grape spikes. Through training, a model recognizes these obstacles in trellis scenes. This information helps dual arms optimize grape bunch positioning, establish harvesting paths, and avoid collisions. Results show the improved model has 60.71 % pixel accuracy and 53.10% intersection ratio, with an 83.77% parameter compression. Segmentation time is 0.09s per image. Accuracy for hard obstacles like branches and stems is 94.90% and 93.64%, and for soft obstacles like branches, leaves, and stalks, it's 92.18% and 85%. In field tests, the dual-arm system harvested at 942spikes/hour with 86.6% fruit harvesting success, less than 5% branch collisions, and less than 5% fruit spike damage. This method enhances the environmental perception capability of the dual-arm grape harvesting robot in the orchard, optimizing the reliability of the dual-arm harvesting process while ensuring its high efficiency.