Self-supervised regrasping using spatio-temporal tactile features and reinforcement learning

We introduce a framework for learning regrasping behaviors based on tactile data. First, we present a grasp stability predictor that uses spatio-temporal tactile features collected from the early-object-lifting phase to predict the grasp outcome with a high accuracy. Next, the trained predictor is used to supervise and provide feedback to a reinforcement learning algorithm that learns the required grasp adjustments based on tactile feedback. Our results gathered over more than 50 hours of real robot experiments indicate that the robot is able to predict the grasp outcome with 93% accuracy. In addition, the robot is able to improve the grasp success rate from 42% when randomly grasping an object to up to 97% when allowed to regrasp the object in case of a predicted failure.