Speech Recognition for a Person With Cerebral Palsy Using Whisper Fine-Tuned on Japanese and English Dysarthric Speech

People with cerebral palsy often have dysarthria, and this makes it hard for them to speak as they wish. In this paper, we present an automatic speech recognition (ASR) model for a person with cerebral palsy based on Whisper. Whisper is highly accurate when performing speech recognition for Japanese speech, but recognition accuracy for Japanese dysarthric speech tends to be low. One possible solution to this problem is to fine-tune Whisper using Japanese dysarthric speech. However, there is a problem in that it is difficult to record a large amount of speech for people with dysarthria. Therefore, in our proposed method, English dysarthric speech is utilized for training Whisper in addition to Japanese dysarthric speech. The results of our proposed method showed an improvement in the error rate of about 1% compared to using only Japanese dysarthric speech as training data.