ESL127: Emirate Sign Language Dataset(s) and e-Dictionary (Version 2.0) Utilizing Deep Learning Recognition Models

As stated by the United Arab Emirates’s (UAE) Community Development Authority (CDA), there are around 3,065 individuals with hearing disabilities in the country. These individuals often struggle to communicate with broader society and rely on scarce sign language (SL) interpreters. Moreover, Arabic’s dialects diversity compounds the issue by causing dialects in the Arabic Sign Language (ArSL). Hence, the call for a standardized reference for ArSL in the region is a priority. To address these challenges, we’ve developed an Emirate Sign Language (ESL) electronic dictionary (e-dictionary) with a dataset of 127 signs and 50 sentences, recorded by hearing-impaired individuals in the UAE with various degrees of deafness. Supervised by certified interpreters and validated by ESL’s department head at CDA in Dubai, the recordings were made using Azure Kinect DK, resulting in 708 recordings. The dataset is then processed to 10fps. The e-dictionary offers features such as webcam-based sign recognition using YOLOv8 technology, voice-based signing via Arabic Automatic Speech Recognition, text-based signing, and words spelling in ArSL.