3-D Tactile-Based Object Recognition for Robot Hands Using Force-Sensitive and Bend Sensor Arrays

Tactile sensing is a particularly important and challenging task for a modern robot to safely manipulate objects, interact with humans in a shared space, and provide various services. This article presents a 3-D tactile glove for robots with the combination of a piezoresistive-based force sensor array (412 sensors) covering the full hand and a resistive bend sensor array (five sensors) on the back of five fingers. Deep learning-based convolutional neural network (CNN) and multilayer perceptron network (MLP) based methods using the designed tactile glove are proposed for object recognition. In the experiment for recognizing 15 objects with a dexterous robot hand, an average classification accuracy of 93.67% has been achieved. Comparison experiments with three other typical classifiers (the Quadratic support vector machine, weighted KNN, and Bagged Trees) and our MLP and CNN methods show an average recognition accuracy of 91.67% with the 3-D tactile glove, revealing an accuracy improvement of 4.17% over only using the force sensor array. We further apply our 3-D tactile glove and the multimodal CNN to identify three other objects and demonstrate their generalization ability of tactile object recognition with an average success accuracy of 78.33%. The proposed 3-D tactile glove can be further used in human–robot interactions, the design of prosthetics and humanoid robots, and for improving the intelligence level in brain–computer collaborative systems.

I. Introduction

Owing to the maturation of robotics itself and the advance in technology, especially in artificial intelligence, robotics has been undergoing major transformations in scope and dimension. From the dominant industrial focus, robotics is rapidly expanding into unstructured human environments. However, new challenges are to be overcome for robots to share the space with humans, while safely interacting with, assisting, serving, and collaborating with them.