Hand Gesture Translation System based on Multi-Sensor Fusion

With the development of sensors and integrated circuits, various sensors are used to detect hand gesture motion data for gesture recognition and broadly utilized in multiple sectors. However, there is a shortage of research on dual-hand gesture recognition, as most existing works focus on single-hand gesture recognition. This paper proposes that a hand gesture translation system based on multi-sensor fusion. We develop a simple data glove that utilizes motion sensors and bend sensors to collect gesture information. We compare and improve multiple algorithms commonly used for gesture recognition. We categorize 62 Chinese sign language gestures into four classes and use different algorithms for recognition based on the gesture type, which improves efficiency. In the end, the Euclidean Distance (ED) algorithm was employed for the identification of static sign language, resulting in an average recognition accuracy of 99.42%. Dynamic sign language of both hands is recognized using a CNN+LSTM model, dynamic sign language was recognized with an average accuracy of 98.61%.