Real-time Fruit Detection Method Based on RGB-D Image Fusion

The automatic fruit picking and yield estimation of agricultural intelligent machinery relies on accurate fruit detection and positioning technology. Since the complex environment of the orchard and the huge size of the existing models hinder the real-time detection of fruits, we propose a real-time fruit detection method based on RGB-D image fusion. The method first uses two extraction modules with the same structure to extract multi-modal features, then makes full use of complementary multi-modal features through an attention fusion module, and finally sends the multi-scale single-input and fusion features to the prediction module to generate the results. In addition, to better fit the apple shape and reduce parameters for lighting the network to meet the requirement of real-time, we replace the traditional rectangular bounding box with a circular bounding box. We verified the deep network model using the KFuji RGB-DS apple dataset and obtained 94.3%, 90.2%, and 94.0% results for AP
⁵⁰
, Precision, and Recall. The accuracy of our method is comparable with those of YOLOv5s and Faster R-CNN in object detection tasks, and our network obviously outperforms them in the number of parameters and reasoning speed. In conclusion, our method reduces the hardware equipment requirements to achieve real-time rates and provide flat accuracy and reasoning speed synchronously.