Recognition of Isolated Gestures of the Russian Sign Language Based on the Component Approach

Russian Sign Language is a language used for communication by people with hearing impairments. Automating sign language translation is an important task in facilitating communication between hearing and deaf people. The paper describes the system of recognition of isolated gestures of the Russian sign language using a component approach. The component approach involves representing a gesture as its components — this paper considers such components as palm configuration, palm orientation and palm position relative to the body. The proposed system first recognizes the individual components of the gesture, based on which the prediction of the gesture sense is made. This makes the system more versatile. We use the MediaPipe framework as a feature extraction tool and LSTM as a classifier. The results show that this approach is very effective on a small set of static gestures. Experiments conducted on a larger set of dynamic gestures indicate that additional features and transformations are required to achieve higher accuracy.