Learning from Transferable Mechanics Models: Generalizable Online Mode Detection in Underactuated Dexterous Manipulation

In this work, we investigate a mechanics-inspired framework for describing fingertip-based planar within-hand manipulation with an underactuated robotic gripper. In particular, this framework leverages fundamental mechanics properties of the hand-object system, including basic terms such as local contact curvature as well as more complex features including the grasp matrix and manipulability metrics. These are extracted using a simple visual approach and then in real-time used for predicting planar manipulation modes: namely rolling, dropped, stuck, and sliding. Given a desired cartesian motion for the object, a supervised learning model predicts these four manipulation modes before they occur, allowing us to either avoid or trigger these different behaviors. Since we utilize strictly fundamental properties of the grasp matrix, finger Jacobians, and contact curvatures, we are able to demonstrate prediction transferability between different grippers using our original classifier. In particular, a Random Forests classifier trained on one gripper successfully predicts manipulation modes for grippers with different fingers with 84% accuracy, compared to just 56% from an approach in previous work. Overall, we find that the features designed in our approach better describes fingertip manipulation when precise gripper models are not available.