Learning a Hand Model from Dynamic Movements Using High-Density EMG and Convolutional Neural Networks

Objective: Surface electromyography (sEMG) can sense the motor commands transmitted to the muscles. This work presents a deep learning method that can decode the electrophysiological activity of the forearm muscles into the movements of the human hand. Methods: We have recorded the kinematics and kinetics of the hand during a wide range of grasps and individual digit movements that cover 22 degrees of freedom of the hand at slow (0.5 Hz) and comfortable (1.5 Hz) movement speeds in 13 healthy participants. The input of the model consists of 320 non-invasive EMG sensors placed on the extrinsic hand muscles. Results: Our network achieves accurate continuous estimation of both kinematics and kinetics, surpassing the performance of comparable networks reported in the literature. By examining the latent space of the network, we find evidence that it mapped EMG activity into the anatomy of the hand at the individual digit level. In contrast to what is observed from the low-pass filtered EMG and linear decoding approaches, we found that the full-bandwidth EMG (monopolar unfiltered) signals during synergistic and individual digit movements contain distinct neural embeddings that encode each movement of the human hand. These manifolds consistently represent the anatomy of the hand and are generalized across participants. Moreover, we found a task-specific distribution of the embeddings without any presence of correlated activations during multi- and individual-digit tasks. Conclusion/Significance: The proposed method could advance the control of assistive hand devices by providing a robust and intuitive interface between muscle signals and hand movements.