Transferring Piano Performance Control across Environments

Player pianos driven by computers are able to record and reproduce various performance control parameters, including pitch, timing, velocity and pedaling. However, the resulting sound of performance is not 100% reproducible in a new environment due to the difference in room acoustics and physical properties of the piano. Inspired by the Psychoacoustic studies which showed that human pianists adjust their controls in new environments for better performances, we have developed a system that automatically transfers performance control across environments in order to make the reproduced sound as similar as the original one. In specific, our work includes (1) a systematic measurement of the control-sound relationship of player pianos under different environments, and (2) a novel algorithm to adjust the control parameters through interpolating the measured control-sound functions. We evaluated the effectiveness of our method by conducting a listening test. Experimental results show that our algorithm outperforms the baseline significantly.

1. INTRODUCTION

It has been discovered by both musicians and scientists [1]–[3] that professional pianists adapt their performance controls such as timing and velocity nuances to different room acoustics and pianos, sometimes even unconsciously, so that the sound effect is consistent across environments. This phenomenon motivates us to design a corresponding mechanism for computer-driven performance systems. In this paper, we present such a system that transfers the control parameters of player pianos across environments so that the difference of perceived sound effect is minimized. The whole procedure is illustrated in Fig. 1. The control C₁ in the source environment E₁ generates Sound 1, while the control C₂ in the target environment E₂ generates Sound 2. We regard the environment E(•) as a function that unifies the factors of room and piano acoustics, mapping the control parameters to actual sound. The goal is to find the optimal C₂ (transferred from C₁) that minimizes the "distance" between Sound 1 and Sound 2. Formally:

$$\begin{equation*}C_2^{\ast} = \mathop {\arg \min }\limits_{{C_2}} \left\{ {dist\left[ {{E_1}\left( {{C_1}} \right) - {E_2}\left( {{C_2}} \right)} \right]} \right\}.\tag{1}\end{equation*}$$