Inference of Grasping Pattern from Object Image Based on Interaction Descriptor

We propose a method of visually recalling appropriate grasping method from the shape of object parts. We quantitatively express the grasping method (i.e. interaction state of hand and object) by introducing its numerical descriptor (interaction descriptor), and recall how to grasp by machine learning of the relationship between partial object shapes and interaction descriptors. Then the relationship between the interaction descriptors and object appearance are learned by CNN regression. Part-based relationship between objects and grasping hand can be modeled and recalled multiple grasping modes for the same object if possible.