SignAR: A Sign Language Translator Application with Augmented Reality using Text and Image Recognition

Studies have shown that hearing impaired students face lots of difficulties in reading comprehension mostly because they cannot develop a fluent system of communication required to become proficient readers. This in turn can have an impact on their communication and social skills, leading to an overall depressed academic achievement, affecting their future prospects. SignAR will help hearing impaired children learn English as well as sign language and hearing people can also use it to learn sign language. So, basically, once a word has been captured by the phone camera, the corresponding sign animation will be augmented on the user’s screen. Each word will be mapped to its corresponding animation and shown on the phone’s screen. By using SignAR, the hearing impaired children will be able to learn English using Signs while hearing users will learn Signs using English. In this way, the communication barriers between the hearing and non-hearing will hopefully be greatly diminished.