Multi-Branch Multi-Scale Channel Fusion Graph Convolutional Networks With Transfer Cost for Robotic Tactile Recognition Tasks

Inadequate consideration of tactile sensor features in existing algorithms can lead to insufficient prediction accuracy for robotic tactile recognition tasks. In this paper, we propose a spatial-temporal information based adjacency matrix construction method called transfer cost. On this basis, we propose a multi-branch multi-scale channel fusion graph convolutional network (MSCF-LSTM-GCN). This network combines the advantages of long short-term memory networks (LSTM) with graph convolutional networks (GCN) to extract spatial-temporal features. It is the first LSTM-GCN network with a multi-branch structure that utilizes multi-feature scale and multi-channel fusion. This structure provides diverse perceptual ranges and information levels. We introduce shortcuts combined with the multi-branch structure to reduce the loss of information caused by multiple layers of transmission. The Einstein summation calculation method and the parameters of binarizing the adjacency matrix are optimized to enable the established dynamic tactile graph to participate in batch training. The proposed method improves both prediction accuracy and F1 score by over 1% on the tactile grasp stability prediction task. In the tactile object recognition task, the proposed method improves both prediction accuracy and F1 score by over 3%.Note to Practitioners—Robotic tactile perception is an important research area that can improve the flexibility, accuracy and stability of robots in robotic tactile recognition tasks. However, the amount of accessible information depends on the resolution and number of tactile sensors, and the features extracted using conventional deep learning methods are limited. For this reason, we propose a spatial-temporal information based adjacency matrix construction method called transfer cost. Then, we propose the MSCF-LSTM-GCN model that extract spatial-temporal features from different feature scales and shortcuts. The network solves the problems of traditional deep learning methods in acquiringsingle features and losing deep information. The experiments on two public datasets show that our method performs due to other competing algorithms on the tasks of robotic grasp stability prediction and object recognition for multi-tactile sensor scenarios. In future research we will invest in the problem of 3D force regression.