A Gesture Recognition System Based on Multi-Modal Physiological Signals

The hand is an important organ of human beings, and hand-losing amputees face enormous pain and inconvenience. Surface electromyography (sEMG) signal is a physiological signal that reflects the intention of human movement and is widely used in prosthetic hand control. However, due to muscle fatigue and electrode movement, there are bottlenecks in using sEMG alone to recognize gestures. This work aims to recognize gestures using multi-modal signals. Based on the Ninapro DB3 dataset, this work extracts features of human upper limb acceleration and sEMG signals, then the principal component analysis method is employed to reduce the dimension. After that, these features are fused to construct seven feature groups, which are classified by three classifiers. The results show that in the vast majority of cases, the accuracy of gesture recognition using multi-modal signals is higher than employing acceleration or sEMG alone. Finally, the acceleration features fuse with the sEMG time-domain features classified by neural network classifier result in optimal recognition accuracy and computation time.