Material Classification through Knocking and Grasping by Learning of Structure-Borne Sound under Changing Acoustic Conditions

Ahstract-Structure-borne sound is an interesting sensory modality for inferring contact information in robotics due to comparatively cheap sensor hardware, the possibility to integrate it into existing systems with little effort, and due to the richness of information in the acquired signals. In this work we investigate whether it is feasible to fit piezoelectric acoustic sensors on a robotic system in order to infer properties of objects during contact events. In contrast to existing works regarding object and material identification by evaluating sound the challenge in our experimental setup is that the sensor is integrated in the structure of the robot. Hence, the measured audio signal are not only governed by the acoustic properties of the objects, but are also strongly influenced by the changing resonance properties of the robot due to its kinematic configuration and the ego-noise during operation. We investigate whether it is possible to learn a classifier that is invariant and robust to these configuration dependent changes in the acquired audio signals. We demonstrate the feasibility of this approach in two selected material classification experiments and compare the performance of a deep learning classifier to several baseline machine learning methods. We found out that a representation learning approach using a deep neural network shows the highest invariance to the changing acoustics properties and outperforms the baseline methods in our experiments. The results encourage the further investigation of structure-borne sound in robotic applications.