A Perception and Alignment Framework for Fruit Harvesting Using Spherical Object Modeling and Hybrid Visual Servoing

Fruit harvesting robots often face challenges in precise target localization, gripper alignment, and adaptability under harsh conditions, especially when fruits are occluded by leaves and stems. Existing solutions often lack transferability due to unique fruit shapes and harvesting methods. To address these issues while maintaining transferability, we propose a perception and alignment framework combining Spherical Object Modeling (SOM) with hybrid visual servoing. SOM quickly approximates fruit position and size, allowing accurate target tracking even under partial visibility. Evaluations demonstrate SOM’s high precision (errors below 4%) and fast processing (under 1 ms), validating its robustness and efficiency. Experiments with melons and tomatoes under various conditions, including different sensor configurations, occlusion, imperfect calibration, and gain selections, demonstrated the adaptability of the framework to various types of fruit and challenges.