Finger tracking for piano playing through contactless sensor system: Signal processing and data training using artificial neural network

Various researches had attempted to unveil the technique of virtuoso pianists using technologies. These researches employ different types of sensors in order to capture motion data of piano playing. Researches that embark on this area faced a common problem, the sensors used in these works are directly touching the pianist, in other words this causes a change of piano playing experience. Since piano playing consists of very delicate interaction between the pianist and the piano, such change of experience may affect the pianist's performance. These sensors are said to have change the piano playing experience of the pianist. Concluding the challenges faced by current technologies, a non-intrusive and long range capacitive sensor is developed. This sensor employs the RC oscillator method where the change of the capacitance is recorded in number of pulses. In this work, a prototype sensor is developed to sense different positions of the fingers on five keys of the piano out of the entire 88 keys. To validate the design, input data with known output position were collected and fed into an artificial neural network for training. The output of the neural network is shown in regression plots, where the overall coefficient of determination, R=0.96747. The fit value and the accuracy is reasonably good for the data set. The output data represents the location of the fingers on the piano keyboard with approximately 10mm deviation.