Research on Real-time Recognition and Classification of Human Gesture Based on BiLSTM-CNN

To improve the classification accuracy of real-time gesture recognition by surface electromyography (sEMG), an inertial measurement unit (IMU) is added to assist the IMU sensor in acquiring the mixed signals of IMU and sEMG. The proposed algorithm is based on a combination of Bi-directional long short-term memory (BiLSTM) and convolutional neural networks (CNN) for real-time gesture recognition and classification. The first layer of the BiLSTM uses the gesture data collected by the IMU for arm orientation recognition. The second layer of the CNN uses the energy kernel phase diagram features for real-time recognition and classification of six gestures with mixed signals of motion states. By optimising the parameters of the model, the BiLSTM-CNN classification algorithm has a real-time gesture recognition time of less than 35ms, an average recognition accuracy of 90.6% for arm direction and 87.3% for gesture classification, which is advantageous in terms of recognition speed and accuracy compared with other machine learning classification algorithms.