A Human Demonstration Based Inverse Dynamics Model for Long-Sequential Dough Shape Manipulation

Long-sequential dough shape manipulation (LSDSM) is essential in food processing and ceramic production. However, it is challenging since high degrees of freedom (DOFs) of dough make it difficult for state representation and planning. Some previous approaches focus on optimizing actions using gradients provided by differentiable particle-based simulators, which still has an obviously gap compared to real scenarios. In this paper, we devise a long roller and present a human demonstration based inverse dynamics model to perform the LSDSM. The proposed model predicts the robot’s actions from human demonstration, which is composed of a roller direction prediction module and a rolling vector prediction module. In the first module, we use a polar form dough representation and an attention-based model trained with 30k virtual data and 0.45k actual data to predict the roller’s direction θ. In the second module, we search the rolling vector using depth maps of dough and θ predicted by the first module. The rolling vector consists of the starting rolling position (x,y) and rolling direction l. We experimentally evaluate the proposed model on twelve representative long-sequential dough shape manipulation tasks: inter-conversions of dough shapes among a circle, a triangle, a parallelogram, and a square. The results show an average 85.41% improvement of similarity with the desired dough shape compared to the initial dough shape. Besides, we conducted several comparative experiments to verify the validity of each part of our proposed model.