StereoTacTip: Vision-Based Tactile Sensing With Biomimetic Skin-Marker Arrangements

Vision-based tactile sensors (VBTSs) stand out for their superior performance due to their high-information content output. Recently, marker-based VBTSs have been shown to give accurate geometry reconstruction when using stereo cameras. However, many marker-based VBTSs use complex biomimetic skin-marker arrangements, which present issues for the geometric reconstruction of the skin surface from the markers. Here, we investigate how the marker-based skin morphology affects stereo vision-based tactile sensing, using a novel VBTS called the StereoTacTip. To achieve accurate geometry reconstruction, we introduce the following: first, stereo marker matching and tracking using a novel Delaunay-triangulation-ring-coding algorithm; second, a refractive depth correction model that corrects the depth distortion caused by refraction in the internal media; third, a skin surface correction model from the marker positions, relying on an inverse calculation of normals to the skin surface; lastly, methods for geometry reconstruction over multiple contacts. To demonstrate these findings, we reconstruct topographic terrains on a large 3-D map. Even though the first and second contributions were developed for biomimetic markers, they should improve the performance of all marker-based VBTSs. Overall, this work illustrates that a thorough understanding and evaluation of the morphologically complex skin and marker-based tactile sensor principles are crucial for obtaining accurate geometric information.