A Systematic Mapping of Computer Vision-Based Sign Language Recognition

Sign languages are the primary mean of communication for the majority of deaf people. Unfortunately, there are multiple barriers due to the lack of assistive technologies, this causes problems in the communication between the Deaf and their environment, access to services, education, information, and employment. Automatic sign language recognition using machine learning techniques has gained special attention in the last years to help reduce such barriers. Different lines of investigation are being followed for isolated and continuous sign language recognition, and multiple models exist that attempt to translate from sign language to a written one. We present a systematic mapping of the papers that have been published relevant to this area between January 2019 and March 2022. The main focus of the study is on technologies used to recognize isolated signs and sentences in different sign languages from RGB videos and without specialized hardware or sensors. A total of 1061 articles are reviewed and classified showing that: (1) There is a majority of publications which use external hardware or recognize alphabet signs from static images, (2) LSTM and different variants of CNNs are the main methods used for isolated sign language recognition and Transformer models are predominant for continuous sign language recognition, (3) Different types of CNNs are the primary feature extraction methods followed by skeleton representations. Finally, we highlight the use of negative vocabulary and labels in research papers to refer to the Deaf community and call for the eradication of this practice.
¹