Translating sEMG signals to continuous hand poses using recurrent neural networks

In this paper, we propose a hand pose estimation approach from low cost surface electromyogram (sEMG) signals using recurrent neural networks (RNN). We use the Leap Motion sensor to capture the hand joint kinematics and the Myo sensor to collect sEMG while the user is performing simple finger movements. We aim at building an accurate regression model that predicts hand joint kinematics from sEMG features. We use RNN with long short-term memory (LSTM) cells to account for the non-linear relationship between the two domains (sEMG and hand pose). Additionally, we add a Gaussian mixture model (GMM) to build a probabilistic model of hand pose given EMG data. We performed experiments across 7 users to test the performance of our approach. Our results show that for simple hand gestures such as finger flexion, the model is able to capture hand pose kinematics precisely.

I. Introduction

Non-invasive surface electromyogram (sEMG) recordings on the forearm contain useful information for decoding muscle activity and hand kinematics [1], [2]. sEMG has been used by researchers to develop intuitive robotic prosthesis interfaces either via pattern recognition using physiological features or via classical control schemes [3], [4]. Classification approaches attempt to estimate hand posture from a pre-defined set using continuous sEMG signals. Classifiers such as support vector machines [1], linear discriminant analysis [5], artificial neural networks [6], fuzzy logic [7], Gaussian mixture models (GMM) [3], among others have been proposed using a wide variety of features (zero-crossings of raw EMG, mean absolute deviation, root mean square of the signal, etc.). However, such approaches are not valid when attempting to fully reconstruct hand kinematics.