Learning Graph Dynamics With Interaction Effects Propagation for Deformable Linear Objects Shape Control

Robotic manipulation of deformable linear objects (DLOs) has broad application prospects, e.g., manufacturing and medical surgery. To achieve such tasks, a critical challenge is the precise control of the DLOs’ shapes, which requires an accurate dynamics model for deformation prediction. However, due to the infinite dimensionality of the DLOs and the complexity of their deformation mechanism, dynamics models are hard to theoretically calculate. In this paper, for representing the DLO, we use multiple particles being uniformly distributed along the DLO. For learning the dynamics model, we adopt Graph Neural Network (GNN) to learn local interaction effects between neighboring particles, and use the attention mechanism to aggregate the effects of these interactions for the purpose of effect propagation along the DLO (called GA-Net). For manipulation, the Model Predictive Control (MPC) coupled with the learned dynamics model is used to calculate the optimal robot movements, which can also generalize to unseen DLOs. Simulation and real-world experiments demonstrate that GA-Net shows better accuracy than existing methods, and the proposed control framework is effective for different DLOs. Specifically, for model prediction (150 steps), the prediction performance of GA-Net is 14.14% better than the strong baseline (IN-BiLSTM). Videos are available at https://parkergu.github.io/work_dlo/. Note to Practitioners—This paper was motivated by the problem of shape control of DLOs (e.g., ropes, cables) but it also applies to other deformable objects. Robotic manipulation of DLOs has broad application prospects across various industries, including medical surgeries and manufacturing. Existing approaches to manipulate DLOs, such as reinforcement learning, suffer from sample inefficiency and challenges in generalization. To alleviate these issues, we propose a model-based framework. We adopt GNN and attention mechanism to learn DLOs’ dynamics. Then we use MPC coupled with the learned dynamics model for manipulation of DLOs. The framework is sample-efficient for manipulation, and can generalize to unseen DLOs. Previous works on GNN-based dynamics model do not consider instantaneous propagation of interaction effects, which leads to a false prediction. To alleviate this issue, we adopt GNN to learn interaction effects between neighboring particles, and use the attention mechanism to propagate local interaction effects along the DLO. Simulation and real-world experiments demonstrate that our dynamics model shows better accuracy than existing methods, and also demonstrate the effectiveness of the proposed control framework.