A Deep Learning-Based Approach to Strawberry Grasping Using a Telescopic-Link Differential Drive Mobile Robot in ROS-Gazebo for Greenhouse Digital Twin Environments

The primary goal of this research is to develop a deep learning-powered robotic solution to address labor shortages and optimize harvesting processes in strawberry greenhouse farms. By incorporating this system into the development process, the aim is to provide continuous, 24/7 operational efficiency for strawberry harvesting in greenhouse environments. This study is grounded in a comprehensive literature review of simulated environments, such as ROS-Gazebo, deep learning detection models, and mobile robot platforms, with a focus on developing innovative robotic solutions for strawberry detection and grasping in a simulated digital twin greenhouse environment. The YOLOv9-GLEAN deep learning model, with super-resolution capabilities, is introduced to enhance strawberry detection accuracy by generating high-resolution image features. We developed a digital twin model of the SILAL strawberry greenhouse farm in Abu Dhabi, UAE, within the ROS-Gazebo environment, to validate our algorithm and test the MARTA (Mobile Autonomous Robot with Telescopic Arm) robot. The dataset used to improve model performance includes both real strawberry images from greenhouse farm and synthetic CAD-generated images. ROS-MoveIt was employed to implement visual servoing, allowing the robot to generate precise motion trajectories to approach and grasp identified strawberries, with visual feedback enhancing accuracy. Empirical results show that our proposed detection model outperforms other existing models, achieving a precision of 0.996 and a recall of 0.991. The model’s adaptability to varied datasets, including real and synthetic images, is notable, and it performs exceptionally well in the simulated digital twin model of the greenhouse farm. The model is uniquely trained on both real and synthetic strawberry images to ensure robust detection performance. It is compared to state-of-the-art models and deployed on a telescopic arm-based robotic platform, which is simpler to control than an articulated arm for strawberry harvesting and grasping tasks.