Adaptive Teaching of the Iranian Sign Language Based on Continual Learning Algorithms

Research has demonstrated that intelligent systems significantly enhance the learning process. This study aims to design and implement an interactive computer-based platform for adaptive teaching of Iranian Sign Language (ISL). Unlike most sign languages that rely solely on hand movements, ISL also requires lip movements, which are crucial for distinguishing many words. This dual requirement presents a unique challenge in detecting ISL words from video frames. To address this, we created a dataset named ISLR101, containing videos of 101 ISL signs. We designed a neural network with three transformer encoder modules for different input features and one for generating output. This architecture enables the system to accurately learn and recognize ISL signs. To ensure the neural network can continuously learn new ISL words without forgetting previously learned ones, we employed the Elastic Weight Consolidation (EWC) method. This approach helps maintain an average accuracy of 82.92% across six training tasks, each comprising approximately 17 classes. Following the training process, we developed an interactive teaching system based on fuzzy logic. This system adapts to users’ needs and performance, enhancing their learning experience. The system’s effectiveness was evaluated using the UTAUT questionnaire in a preliminary exploratory study involving 20 hearing individuals (10 males and 10 females). The results indicated that the adaptive teaching architecture interacted effectively with different users (Cohen’s d = 0.57) and adapted to them (Cohen’s d = 0.63) over four training sessions. Additionally, users showed increased motivation to interact with the intelligent educational system (Cohen’s d = 0.92).