Detection and Location of Litchi Fruit based on Object Detector and Depth Image

Computer vision is the core of harvesting robots, and computer vision-based litchi fruit detection is an important direction for research on automated harvesting. However, the current detection methods of litchi fruit cannot meet the detection requirements in the complex orchard environment and cannot simultaneously detect a single litchi fruit and calculate the spatial location of the fruit. In this paper, a detection and location method based on object detector and depth images was proposed in this paper for litchi fruit in an orchard environment. After comparing the performance of multiple object detectors, YOLOv5 was chosen as the base model for litchi detection. An attention function and an improved loss function are added to YOLOv5. Meanwhile, to transmit more shallow features to deep layers, the transmission path of the neck was adjusted. The experimental results show that the mAP and F1 scores of the improved YOLOv5 are 0.9534 and 0.9428, which are higher than the corresponding performance values of YOLOv5, and the improved YOLOv5 can overcome the influence of various factors on the detection performance in practical detection scenarios. The method was tested in the orchard environment Intel RealSense D435 was used to collect RGB images and depth images of litchi during the test. The improved YOLOv5 detected litchi fruits in RGB images, the values of their corresponding positions in depth images were read. The experimental results show that the proposed method has high accuracy and strong robustness.