A Novel Multi-task Learning based Automatic Speech Impairment Assessment Algorithm

Speech impairment assessment is crucial to the treatment evaluation of speech therapy. The current evaluation methods mainly depend on speech-language pathologists (SLPs). Automatic speech impairment assessment (ASIA), especially the regression of severity scores, has not received enough attention. So we present a novel ASIA algorithm which based on the multi-task leaning with joint severity level classification and score regression. Owing to the auxiliary classification task, the precision of the severity score prediction can be improved effectively. In addition, the residual network (ResNet) and the long short-term memory (LSTM) are cascaded as the backbone. The performance of the model is demonstrated on the Mandarin AphasiaBank dataset and the experiments show that the algorithm achieves promising performance.