AllTact Fin Ray: A Compliant Robot Gripper With Omni-Directional Tactile Sensing

Tactile sensing plays a crucial role in robot grasping and manipulation by providing essential contact information between the robot and the environment. In this paper, we present AllTact Fin Ray, a novel compliant gripper design with omni-directional and local tactile sensing capabilities. The finger body is unibody-casted using transparent elastic silicone, and a camera positioned at the base of the finger captures the deformation of the whole body and the contact face. Due to the global deformation of the adaptive structure, existing vision-based tactile sensing approaches that assume constant illumination are no longer applicable. To address this, we propose a novel sensing method where the global deformation is first reconstructed from the image using edge features and spatial constraints. Then, detailed contact geometry is computed from the brightness difference against a dynamically retrieved reference image. Extensive experiments validate the effectiveness of our proposed gripper design and sensing method in contact detection, force estimation, object grasping, and precise manipulation. Note to Practitioners—Fin Ray fingers have been widely adopted for robot grasping and manipulation due to their adaptive nature. However, existing Fin Ray finger designs lack tactile sensing capabilities, or can only perceive contacts at certain regions. The motivation of this work is to enable the Fin Ray gripper to not only detect contacts in all directions but also measure the detailed contact geometry of the grasped object. The omni-directional contact sensing ability is crucial for manipulation in cluttered and dynamic scenarios. And the contact geometry measurement is critical for in-hand pose estimation and precise manipulation. The proposed finger design is only composed of a single transparent elastic silicone cast, a camera and white LED, making the manufacturing process simple and time-efficient. The design files and sensing algorithms will be open-sourced to foster further development. We validate the effectiveness of this sensor design and sensing method across various applications. It can locate contact positions with an error less than 1mm, accurately estimate the force magnitude on the contact surface, and detect contact forces from different directions with a threshold below 2 N. In future research, we will explore advanced manipulation applications including deformable object and bi-manual manipulation and improve the finger’s durability.