Phoneme Embeddings on Predicting Fundamental Frequency Pattern for Electrolaryngeal Speech

Electrolaryngeal (EL) speech has robotic quality owing to constant fundamental frequency (F
₀
) patterns. In existing F
₀
pattern prediction frameworks, acoustic models are trained on spectral features of a large corpus of healthy speech. However, EL speech does not embed any useful information about F
₀
into spectrogram. Moreover, creating datasets with reasonably large number of EL utterances for training neural networks is very time-consuming. Hence, F
₀
prediction based on other features with sharing capability between EL and normal speech must be investigated. In this study, we investigate F
₀
prediction based on clustering of the phoneme embeddings. For a dataset consisting of utterances of both speech types, phoneme labels are extracted. These phoneme labels are then used to learn phoneme embeddings in a common 2-D space. Through clustering of the learned phoneme embeddings, new onehot features are created for F
₀
prediction. Experimental results show that when considering training sets consisting mixed utterances of EL and normal speech, by using new features, improvements in F
₀
prediction accuracy can be achieved. Moreover, accurate F
₀
patterns can be predicted even based on lower-dimensional features corresponding to small values for the number of clusters. This could simplify the structure of the recognition system required to extract phoneme labels from EL speech.