Integrated Object, Skill, and Motion Models for Nonprehensile Manipulation

Advanced hand skills for object manipulation can greatly enhance the physical capability of robots in a variety of applications. Models that can comprehensively and ubiquitously capture semantic information from the demonstration data are essential for robots to learn skills and act autonomously. Compared to object manipulation with firm grasping, nonprehensile manipulation skills can significantly extend the manipulation ability of robots but are also challenging to model. This paper introduces several new modeling techniques for nonprehensile object manipulation and their integration for robot learning and control. Other than a basic map of the object’s state transitions, the proposed modeling framework includes a generic object model that can help a learning agent infer manipulations that have not been demonstrated, a contact-based skill model that can semantically describe nonprehensile manipulation skills, and a motion model that can incrementally identify patterns from crowdsourced and constantly collected data. Examples and experiment results are given to explain and validate the proposed methods.