On-line real-time fruit size estimation using a depth-camera sensor

Fruit weight is one of the factors taken into account when performing yield estimations together with the trees density and orchard's area. Thus, having the possibility to collect data about the weight of a large number of fruits in the orchard gives the possibility to increase the reliability of the yield estimation. Over recent years, mathematical models able to convert the fruit size into fruit weight were evaluated as effective. Since then, manual data collection with calipers and automated/continuous fruit gauges were tested to collect fruit size data to perform yield predictions. Their main drawbacks are respectively the need for human-labour, repetitiveness, being time-requiring and the limited sample varying from 20 to 200 fruits per hectare. This research is trying to discover and deepen the use of AI in agriculture for doing a step further: sizing fruits after their detection with a YOLOv5 Neural network algorithm. To reach this goal, a system which takes as a input RGB-D depth-camera's color images and 16 bit depth maps was developed. After applying YOLOv5 detection, two different methodologies (by mean of squared bounding boxes and circular shapes) to extract from the depth map the distance data needed to size the target object were tested. Results from a preliminary data-set showed that the system could be a potential solution to increase the sample dimension and perform yield prediction. The main drawbacks of the developed vision-system are related to the errors in sizing the objects, which are ranging from an underestimation of about 9 mm to an overestimation of 24 mm. From the initial results was possible to identify the squared-bbox-mediated sizing process as a better pathway rather than the one performed with circular-bboxes, since the RMSE is always smaller with values of 7–9 mm