DiGeTac Unit for Multimodal Communication in Human–Robot Interaction

Sensing technologies are required to enhance the safety and interaction between humans and robots in the concept of human–robot collaboration (HRC). In this letter, a multimodal sensing unit, i.e., DiGeTac, composed of two layers with distance, gesture, and tactile elements, is presented. Distance and gesture elements are placed on the top layer of the DiGeTac, while the tactile element is placed on the bottom layer. The sensing unit DiGeTac and its data acquisition board together are designed as a sensing module. The sensor performance for detection of distance, hand gestures and touch perception are analyzed with offline and real-time experiments. The sensing module recognizes hand gestures with 96% accuracy and contact location with 88% accuracy using an artificial neural network and a convolutional neural network, respectively. The sensing capabilities of the proposed module are validated in real-time with a collaborative task using the Universal Robot arm. The sensing capabilities of the proposed DiGeTac, as demonstrated in real-time collaborative tasks with the robot arm, highlight its potential to enhance the interaction in HRC settings by effectively recognizing and responding to human commands.