Exploiting Visual-Outer Shape for Tactile-Inner Shape Estimation of Objects Covered with Soft Materials

In this letter, we consider the problem of inner-shape estimation of objects covered with soft materials, e.g., pastries wrapped in paper or vinyl, water bottles covered with shock-absorbing fabrics, or human bodies dressed in clothes. Due to the softness of the covered materials, tactile information obtained through physical touches can be useful to estimate such inner shape; however, using only tactile information is inefficient since it can collect local information at around the touchpoint. Another approach would be taking visual information obtained by cameras into account; however, it is not straightforward since the visual information only captures the outer shape of the covered materials, and it is unknown how much such visual-outer shape is similar/dissimilar to the tactile-inner shape. We propose an active tactile exploration framework that can utilize the visual-outer shape to efficiently estimate the inner shape of objects covered with soft materials. To this end, we propose the Gaussian Process Inner-Outer Implicit Surface model (GPIOIS) that jointly models the implicit surfaces of inner-outer shapes with their similarity by Gaussian processes. Simulation and real-robot experimental results demonstrated the effectiveness of our method.