Classification of Visual-Tactile Fusion Object Attributes based on the GAMP Attention Mechanism

To effectively improve the robot’s ability to perceive the external environment. This paper proposes a visual-tactile fusion object attribute classification method based on the GAMP attention mechanism. The method first consists of DenseNet169 and Tactile-5Net, which are used as visual-haptic feature extractors, respectively, and the extracted feature vectors are inputted into the GAMP attention mechanism to obtain attention. Then, the feature-level fusion method is adopted to perform feature fusion to get the visual-tactile fusion vectors, and finally, the attribute classification is carried out. Experiments show that the proposed classification method has a classification accuracy (AUC value) of 0.9886 for physical attribute adjectives and 0.9756 for objects on the VHAC dataset, which verifies the effectiveness of the proposed method.