Fine-Tuning Pre-Trained Audio Models for Dysarthria Severity Classification: A Second Place Solution in the Multimodal Dysarthria Severity Classification Challenge

Dysarthria, a neurological disorder affecting speech, poses significant challenges for automatic diagnosis and severity classification due to its complexity and the scarcity of relevant datasets. This study addresses these challenges by leveraging the dataset provided by the ISCSLP 2024 Multimodal Dysarthria Severity Classification Challenge. Our approach primarily involves two key steps: splitting the training set into training and validation subsets, and fine-tuning pre-trained au-dio models to adapt them to the task. To ensure robust evaluation and prevent label leakage, we employed the StratifiedGroupKFold function from sklearn for dataset splitting, ensuring that training and validation sets do not share participants. This method allowed us to maintain consistent statistical distributions across folds. For model finetuning, we selected two pre-trained audio models, w2v-bert-2.0 and whisper-Iarge-v3, and fine-tuned them on different folds of the dataset. Our results indicate that the w2v-bert-2.0 model fine-tuned on fold-2 achieved the highest performance, with a validation Fl-score of 0.699 and an online score of 7.7452. The experimental results, including confusion matrices and embedding distributions, highlight the model's ability to distinguish between different severity levels of dysarthria. Our approach achieved a commendable second place in the competition’ demonstrating the effectiveness of fine-tuning pre-trained audio models for this task. Future work will explore the integration of multimodal data and multi-task learning strategies to further enhance model performance.