Designing Soft-Hardware Complex for Gesture Language Recognition using Neural Network Methods

In recent years, the problem of social adaptation of people with disabilities has been actively solved using developments including artificial intelligence methods. The article describes elaboration of a software and hardware complex (SHWC) for recognizing the sign language of disabled people with hearing and speech impairments. The analysis of analogue products for automatic sign language translation is carried out, technical and design requirements for the soft-hardware complex are formulated, the architecture of the SHWC, the functionality of the server and user application subsystems are described. The design and implementation of a subsystem for the collection and processing of photo and video materials with elements of sign language, including static and dynamic fingerprints, gestures, simple phrases, was carried out. The structure of the file data storage and metadata base has been developed. Scenarios and algorithms for sequencing and transforming video data are described. The sequence of data preprocessing when forming a training set using the augmentation method is described. A model for detecting hands in an image is described.