Sentiment Analysis in Turkish Sign Language Through Facial Expressions and Hand Gestures

In sign languages, where communication is achieved through hand gestures, facial expressions and body language, signs are the subject of many studies due to the diversity in terms of the position of different body parts. These diversities are also encountered in emotion detection in Turkish Sign Language (TID), making direct translation of hand gestures inadequate for emotion detection. Accordingly, in this study, for the first time in the literature, sentiment analysis in TID was performed using facial expressions and hand gestures. For this purpose, a specialized model for the tasks of emotion extraction from facial expressions and gesture recognition from hand gestures was fine-tuned with the dataset collected in this study. As a result, facial expressions are found to be more significant than hand gestures in sentiment analysis in TID, but when supported with hand gestures, the performance improved even more.