Multi-Modal Dyslexia Detection Model via SWIN Transformer With Closed-Form Continuous Time Networks

Dyslexia is a condition that reduces the cognitive abilities of the individuals. Dyslexic individuals (DI) face challenges in recognizing letters and words. Healthcare centers employ multiple modalities, including functional magnetic resonance imaging (MRI), Electroencephalogram (EEG), eye movements, and handwritten images for dyslexia detection (DD). A limited number of research studies employed multi-modalities for DD. The existing DD models demand substantial computational resources to identify dyslexia patterns. Ensemble learning (EL) approaches integrate the efficiency of multiple models to produce an optimal outcome. In this study, the authors developed an EL-based DD model for detecting dyslexia using MRI and EEG. An improved SWIN transformer was used to extract features from MRI images. A closed form continuous time neural networks (CFC) model was fine-tuned to detect crucial patterns from EEG data. The authors employed CatBoost and Dartbooster XGBoost (DXB) as base models for dyslexia prediction. An extremely randomized tree was used as meta-model to make a decision using base models outcomes. The model was developed based on two MRI datasets with an EEG dataset. The authors compared the proposed DD model’s performance against single and EL-based DD models. The findings indicated a significant improvement in DD using the model. The proposed model delivered an exceptional performance by achieving 98.5% and 98.7% on MRI and EEG datasets. Educational institutions and healthcare centers can benefit from the proposed DD model by rendering better services to DI. Generalizing the proposed model in diverse datasets can improve its performance. The proposed DD model’s performance can be enhanced using genome data.