Learning Task-Oriented Dexterous Grasping from Human Knowledge

Industrial automation requires robot dexterity to automate many processes such as product assembling, packaging, and material handling. The existing robotic systems lack the capability to determining proper grasp strategies in the context of object affordances and task designations. In this paper, a framework of task-oriented dexterous grasping is proposed to learn grasp knowledge from human experience and to deploy the grasp strategies while adapting to grasp context. Grasp topology is defined and grasp strategies are learned from an established dataset for task-oriented dexterous manipulation. To adapt to various grasp context, a reinforcement-learning based grasping policy was implemented to deploy different task-oriented strategies. The performance of the system was evaluated in a simulated grasping environment by using an AR10 anthropomorphic hand installed in a Sawyer robotic arm. The proposed framework achieved a hit rate of 100% for grasp strategies and an overall top-3 match rate of 95.6%. The success rate of grasping was 85.6% during 2700 grasping experiments for manipulation tasks given in natural-language instructions.