Humanoid Pianist: Dexterous Piano Playing with Synergy-based Hand Representation*

Playing the piano with dexterous hands has long presented unique challenges for robotics research due to the complex dexterity required. Traditional approaches utilizing individual control of each finger and joint in a robot hand make achieving fluent musical performances difficult and introduce significant control complexity due to the high degree-of-freedom nature of the dexterous hand workspace. This paper introduces a system that explores dexterous piano playing more naturally through a synergistic approach. The system employs a dual-armed robot with hands designed around two main synergies governing coordinated finger movements, which allows defining simple poses to represent both single-note and double-note playing. A mapping connects specific musical notes to the corresponding hand poses and precise timing required for a realistic piano rendition. A robot operating system is utilized to allow complex sequences and rhythms to be concisely specified. Results show that a synergistic approach is feasible and allows convenience with considerable accuracy and repeatability.