Integrating Vision and Learning-based Method for Dexterous Grasping

Robotic grasping and manipulating objects with varying properties pose significant challenges. This paper presents an approach integrating vision-based object detection using You Only Look Once (YOLO, version 8) with learning from demonstration (LfD) for dexterous robotic grasping. The YOLOv8 network accurately detects and localizes objects, providing 3D coordinates for positioning a dexterous hand. A Long Short-Term Memory (LSTM) network, trained on teleoperated data, learns to adaptively adjust grasping forces and finger positions based on object properties and real-time feedback. Experimental results demonstrate high success rates in securely grasping diverse objects while modulating forces within safe limits, achieving human-like dexterity. Integrating vision, learning techniques, and adaptive force control enables enhanced robotic automation while minimizing product damage.