Robotic Fruit Harvesting With Dual Camera System

One challenge facing the world today is the increasing demand for food while the availability of labor decreases in the agricultural sector. The use of robots and automation is a potential long-term solution that may support labor shortage. The Robotics Vision Lab at Northwest Nazarene University has developed the Orchard Robot (OrBot), an innovative solution to assist farmers in orchard management. OrBot is equipped with an articulated manipulator, a dual machine vision system, a customizable end effector, a mobile platform, and an onboard computer with AI capabilities. Initially, the configuration of OrBot used only a single vision system to locate the fruit and guide the robot to pick the fruit. However, the single vision system mounted on the end effector had a limited field of view. This paper describes the addition of a second vision system to increase the field of view, find more fruits, and improve harvesting. The control system of OrBot was developed using ROS 2, a middleware platform that facilitates robot control. All components of OrBot were assigned nodes: the two vision sensors, manipulator, end effector, mobile tank, and image processing. A master node was created to coordinate all the different nodes through topics, services, and actions to perform the harvesting task. OrBot was tested indoors using artificial trees and apple fruits. The artificial fruits were positioned randomly on the two trees. OrBot had a 100 percent picking success rate for all trials and an average of 14 seconds per fruit cycle time. OrBot was then tested on a commercial orchard in 2024 during Pink Lady apple harvest season. OrBot had a success rate of about 75 percent.