Using Human-Computer Interaction (HCI) and Artificial Intelligence (AI) in Education to Improve the Literacy of Deaf and Hearing-Impaired Children

In contrast to hearing-impaired children and deaf children born to deaf parents, deaf children born to hearing parents have limited exposure to language continuously, which impairs their working memory. Using artificial intelligence (AI) and human-computer interaction (HCI), we designed a children’s picture book for helping deaf and hearing-impaired children become literate in both sign language and spoken language. Our goal was to establish a technological environment that is more inclusive by enhancing the functionalities of sign language-dependent young children’s senses. By combining sign language and spoken language in the picture book, we aimed to bridge the communication gap between deaf and hearing-impaired children and their peers. The AI and AR-driven solution not only facilitated language learning but also fostered social inclusion by enabling seamless interaction with both sign language users and non-sign language users. The objective was to stimulate extensive dialogues regarding the authentic and efficient collaboration between educators and researchers, as well as future educators, in order to improve academic achievements for young deaf students, as well as to stimulate comprehensive discussions regarding how to involve educators in the planning and creation of educational digital content.