Tactile Classification of Object Materials for Virtual Reality based Robot Teleoperation

This work presents a method for tactile classification of materials for virtual reality (VR) based robot teleoperation. In our system, a human-operator uses a remotely controlled robot-manipulator with an optical fibre-based tactile and proximity sensor to scan surfaces of objects in a remote environment. Tactile and proximity data and the robot's end-effector state feedback are used for the classification of objects' materials which are then visualized in the VR reconstruction of the remote environment for each object. Machine learning techniques such as random forest, convolutional neural and multi-modal convolutional neural networks were used for material classification. The proposed system and methods were tested with five different materials and classification accuracy of 90 % and more was achieved. The results of material classification were successfully exploited for visualising the remote scene in the VR interface to provide more information to the human-operator.