BioPoint: Enhancing Human-Computer Interaction through Single-Site, Multi-Sensor Gesture Recognition

Advancements in human-computer interaction (HCI) and machine learning are seen as key avenues to help individuals living with upper limb disabilities in accomplishing their activities of daily living. Multi-channel myoelectric systems are a promising approach for HCI due to their intuitive and accurate capture of user intent through muscle activity. However, such systems are still bulky compared to widely accepted smartwatches-like devices and as such pose a challenge for seamless integration into daily life. Consequently, this work presents the first single-site EMG-IMU-PPG system for multi-gesture recognition from the wrist. Overall, when evaluated on eight able-bodied participant, the system achieves an average accuracy of 65.28%±10.45% on 17 finger and wrist gestures and 90.98%±5.65% on eight hand gestures. This study also introduces "Myo Hero", an online environment inspired by "Guitar Hero." Myo Hero provides a dynamic method for testing multi-gesture recognition systems, improving the evaluation process’s appeal. Online evaluation of the multimodal system substantially outperformed random play, signifying EMG-IMU-PPG’s potential for human-computer interaction.