Classification of Dynamic Hand Gestures using Multi Sensors Combinations

Medical applications in general and hand gestures classification-based rehabilitation applications specifically are highly dependent on the choice of essential parameters such as the set of gestures and the number and type of used sensors among others. Aiming for medically satisfying results, the choice of the artificial intelligence algorithm to implement, and the choice of the data set to investigate, are critical to making. Different types of signals can be collected while preparing the data set, such as force myography (FMG), surface electromyography (sEMG), and body’s specific force, angular rate, and orientation, using a combination of accelerometers, gyroscopes, and magnetometers as Inertial Measurements Units (IMU). To ensure dynamic hand gesture identification, this paper proposes to investigate the combination of these three types of channels in one embedded system placed on the subject’s hand. The whole system contains two sEMG sensors, four strain sensors in one hand glove, eight pressure sensors fixed in one armband, and one 9 axis IMU sensor placed on the hand. Ten different dynamic hand gestures, collected from one healthy participant, have been performed twenty times using the left hand. For the classification process, three machine learning approaches namely k-Nearest Neighbors (kNN), Decision Tree (DT), and Random Forest (RF) are implemented. Different combinations of sensors from the system are investigated to test their relevance for the collected gestures classification. The algorithms are evaluated based on the classification accuracy, training time, and testing time and used as the sensor relevance indicator. Results show that using the whole system results in the best accuracy up to 96.12% and 99.55% with DT and RF, respectively.