The Impact of Keypoints Normalization on SIBI Recognition using Modified Shift-GCN

Sign language is a form of non-verbal communication usually used by someone who is speech impaired or deaf to communicate with others. There have been many efforts to develop sign language recognition. However, the dataset is mainly limited to images containing the hand area with static poses. In a previous study, Modified Shift-Graph Convolutional Network (Modified Shift-GCN) performed well on American Sign Language Recognition (ASL) with a relatively simple graph dataset without tracking and normalization. In this study, we explore the implementation of Modified Shift-GCN to recognize the Indonesian Sign Language (SIBI) on a more complex graph dataset combining the hand and body graphs with dynamic poses. We investigate the effect of key-point tracking and normalization on the performance of the Modified Shift-GCN on sign language recognition. The experiments using 10-folds cross-validation show that the normalization yield an average accuracy of 97.9
^%
, superior to that without normalization with an average accuracy of 55.3%.