Virtual Piano Dynamics Enhancement with Electromyography

Pianos are musical instruments that are not easily accessible and portable, which can cause limitations in learning the instrument. Although various technologies have shown promising results in resolving this issue, an optimal solution that can overcome the physical limitation yet provide reality and immersive playing experience has yet to be achieved. This research proposed a lightweight and portable virtual piano solution for beginner-level piano playing, enhanced with biofeedback and tactile sensation to overcome the limitation of lacking in dynamics in most of the virtual piano application. The player's hands are detected and tracked for playing movements as they interact with the 3D graphical interface by using the Leap Motion Controller (LMC). The MyoWare muscle sensors are used to detect the electromyography (EMG) readings to enable the dynamic variation while playing. User testing was carried out on two groups of participants: experienced and novice players respectively. Quantitative and qualitative data have been collected for accuracy, usability, workload, and realism analysis. Results of the study show that this approach has the potential to be used as a beginner's instrument for simple pieces or warm-up practices. The results also indicate that with the addition of tactile sensation by using an acrylic panel and incorporating dynamics with the EMG signals into the solution has led to a better immersive user experience. This study provides an alternative option and worthwhile insight for future development of virtual piano applications.