Multi-culture Sign Language Detection and Recognition Using Fine-tuned Convolutional Neural Network

Communication through hand gestures is an active research topic in computer vision and Human Computer Interaction (HCI). Deaf and speech-impaired community may benefit from the advancement of sign gesture recognition as an intermediary visual communication technique. There is no universal sign language, but different nationalities have their own sign languages. Usually sign gestures ought to be distinct and independently distinguishable and researchers in this domain have always focused on techniques for a particular sign language. However, recognizing sign languages from multiple nations could be helpful in various situations such as dealing with multiple signing persons without knowing their nationalities. This paper presents a novel approach where a two-phased system is designed based on deep learning for identification of the sign language and recognition of the sign gesture. This approach opens a new research direction for the advancement of multi-sign language development. The proposed approach has been implemented and evaluated using two custom-based CNN models and compared to some other popular CNN models. More than 98% accuracy can be achieved when tested on two different datasets.