Interaction Control for Tool Manipulation on Deformable Objects Using Tactile Feedback

The human sense of touch enables us to perform delicate tasks on deformable objects and/or in a vision-denied environment. To achieve similar desirable interactions for robots, such as administering a swab test, tactile information sensed beyond the tool-in-hand is crucial for contact state estimation and contact force control. In this letter, a tactile-guided planning and control framework using GTac, a heteroGeneous Tactile sensor tailored for interaction with deformable objects beyond the immediate contact area, is proposed. The biomimetic GTac in use is an improved version optimized for readout linearity, which provides reliability in contact state estimation and force tracking. A tactile-based classification and manipulation process is designed to estimate and align the contact angle between the tool and the environment. Moreover, a Koopman operator-based optimal control scheme is proposed to address the challenges in nonlinear control arising from the interaction with the deformable object. Finaly, several experiments are conducted to verify the effectiveness of the proposed framework. The experimental results demonstrate that the proposed framework can accurately estimate the contact angle as well as achieve excellent tracking performance and strong robustness in force control.