Depth image-based deformation estimation of deformable objects for collaborative mobile transportation

Human-Robot collaborative transportation is a promising technology that combines the strength of humans and robots. The most common approaches rely on methodologies that exploit force-sensing. However, the drawbacks are multiple. First, the magnitude of force applied might be limited to avoid damages. Then, force measurements might be unidirectional according to the material properties; e.g., compression forces are not measurable for fabrics. This paper proposes an approach based on the estimation of the deformation state of the manipulated object from depth images. Specifically, the segmented depth image of the manipulated object are fed to a Convolutional Neural Network (CNN) model to estimate the current deformation status. Compared with the desired deformation, the current deformation status is used to generate the robot’s twist command. The methodology is proved in a mobile robot application, where carbon-fiber fabrics are transported. A comparison with the state-of-the-art is reported proving that the proposed method is more accurate and more repeatable.