Deep Learning-based Object Understanding for Robotic Manipulation

Manipulation of objects by a robot arm requires an understanding of the various properties of the object. The robot needs a lot of information for object manipulation, there are few algorithms to estimate such information simultaneously. In this study, we propose an object understanding network (OUNet) based on deep learning that simultaneously estimates three key properties for robot object manipulation: object state, contact position for object manipulation, and manipulation type. The object state means whether an openable object is open or closed. The contact position and manipulation type for manipulating objects means where and what the robot should do to change the object state. Usingthis information, it is expected that the robot will be able to select the appropriate manipulation for the current situation of the given object. Experiments were conducted to verify the performance of the OUNet, and it was shown that three key properties can be successfully detected.