Dynamic Object Grasping in Human-Robot Cooperation Based on Mixed-Reality

Static Object grasping is a challenging task that has been studied for decades. The difficulty of the task comes back to the reason that a grasping attempt can have many solutions or due to the uncertainty about the targeted object’s features and characteristics. This makes the fact about dynamic object grasping with un-modeled dynamics even more challenging. In this paper, an approach for dynamic object grasping is presented. The approach considers human-robot handover operation where the robot should be able to track human’s holding-object hand and plan a successful grasp of the object in hand. The system was implemented with the help of Mixed-Reality using HoloLens glasses for human’s hand tracking. A serial manipulator was used to execute the operation mounted with end-effector-mounted camera to perform computer vision operations for grasp planning and correction. The main task is robot at random configuration can be able to find hand-holding object and plan grasp on object in hand. The implemented system shows success and was able to perform most of the grasping tasks successfully.