Online audio beat tracking for a dancing robot in the presence of ego-motion noise in a real environment

This paper presents the design and implementation of a real-time real-world beat tracking system which runs on a dancing robot. The main problem of such a robot is that, while it is moving, ego noise is generated due to its motors, and this directly degrades the quality of the audio signal features used for beat tracking. Therefore, we propose to incorporate ego noise reduction as a pre-processing stage prior to our tempo induction and beat tracking system. The beat tracking algorithm is based on an online strategy of competing agents sequentially processing a continuous musical input, while considering parallel hypotheses regarding tempo and beats. This system is applied to a humanoid robot processing the audio from its embedded microphones on-the-fly, while performing simplistic dancing motions. A detailed and multi-criteria based evaluation of the system across different music genres and varying stationary/non-stationary noise conditions is presented. It shows improved performance and noise robustness, outperforming our conventional beat tracker (i.e., without ego noise suppression) by 15.2 points in tempo estimation and 15.0 points in beat-times prediction.

I. Introduction

In order to introduce the ability of music listening and cognition into musical expressive robotic agents, which simultaneously generate corporeal motor responses, robot audition algorithms must consider causal and low-cost computations. Besides, they must cope with the signal distortions caused by environmental and robot's ego noises (i.e., motor noises generated during the robot's motion), since these degrade the performance of Music Information Retrieval (MIR) algorithms at the audio signal level.