RNN and Mel-spectrogram based acoustic balancing and bouncing of ping pong ball

This paper basically addresses real time neural network control of a peculiarly mechanical system using multi-channel audio input. The aim distinctly is to balance a bouncing ball on a wooden frame by taking into account the acoustic feedback from sensors mounted on the corners of the wooden frame. We first considered the audio data from sensors at an initially known position to create a Mel-Spectrogram that results as the sole input of a recurrent neural network. In our application, the essential subtle differences between the spectrograms from multiple sources are used to drive the position of actuators which determine the position and rotation of a parallel manipulator. We present a real time simulated implementation of this method which operates under physically accurate conditions.