Stuttering Detection Using LSTM and LSTM-Attention Based Convolutional Neural Network

Stuttering is a common speech disorder which results in involuntary repetitions, prolongations, blocks or interjections while speaking. Detection and evaluation using traditional methods is time-consuming. Therefore, we conduct a study on 2 classification based deep learning models for stuttering detection: 1) a model which comprises of a Convolutional Neural Network (CNN) followed by Long-Short Term Memory (LSTM) and Attention layers 2) a model which comprises of a CNN followed by LSTM layer only. We used the SEP-28K dataset for training and evaluating our models. These models take mel spectrogram as input and perform binary classification for each disfluency label of the SEP-28K dataset. The CNN + LSTM + Attention model achieved a higher accuracy of 70.47% in detecting prolongation, 60.80% in block and 56.90% in word repetition compared to the CNN + LSTM model, which achieved 69.43%, 60.34% and 55.23% for the same labels, respectively. However, the CNN + LSTM model outdid the attention-based model in classifying fluent/no-stutter instances with 66.52% accuracy against 61.50% and sound repetition with 65.22% accuracy against 64.25%. For interjection, both the models had similar performance in terms of accuracy. For F1 scores, the CNN + LSTM + Attention model showed improvement in block, prolongation, sound and word repetition classifications, while the CNN + LSTM model observed better results for fluent/no-stutter and interjection detection.