Enhancing Medical ASR Accuracy through the Integration of T5-Based Small Language Models and Error Correction Mechanisms

A key challenge in healthcare is the time spent on documentation, which hinders patient care. Automatic Speech Recognition (ASR) technology automates this process and enhances efficiency and access to medical data. Medical ASR systems face significant challenges in accurately understanding the complexities of healthcare conversations, including various accents, dialects, and specialized terminology. Fine-tuning ASR systems with domain-specific medical speech datasets can improve performance, but the challenge of acquiring large and high-quality datasets limits accuracy. To tackle this issue, we propose an ASR output error correction method using a T5-based language model, which integrates the strengths of large language models with error correction training to improve ASR performance. The model was fine-tuned using a diverse dataset comprising synthetic text, synthetic speech, and natural speech data. Synthetic text data was generated to simulate common transcription errors, producing $\mathbf{3 0, 0 0 0}$ pairs of incorrect and correct text. For performance evaluation, synthetic speech recordings were created from 5,000 text samples using edge text-to-speech synthesizer. Additionally, natural speech data was sourced from the Medical Speech, Transcription, and Intent dataset, consisting of $\mathbf{6, 6 0 2}$ speech recordings focused on medical symptoms. Experiments were conducted to assess the model’s accuracy and Word Error Rate (WER) across these datasets. The T5-based error correction model significantly improved ASR performance, achieving an accuracy of $91.23 \%$ and a Word Error Rate (WER) of just $1.20 \%$ on synthetic text data, while also enhancing WER to $3.03 \%$ for synthetic speech and $\mathbf{3. 6 4 \%}$ for natural speech after targeted training. To evaluate the effectiveness of our models for Indian doctors and their specific context, we recorded prescription speech from six Indian doctors. The experiments on these recordings demonstrated that targeted training reduced the WER from $\mathbf{1 9. 2 5 \%}$ to $\mathbf{8. 8 9 \%}$.