Machine Learning Applied in Speech Science

We study the qualitative and quantitative information acquired from two-dimensional dynamic audiovisual sources storing synchronized image and sound signals recorded during human speech. Our main tool is machine learning, which connects data arising from records made by ultrasound (US) and magnetic resonance imaging (MRI) techniques. As a starting point, we track the tongue contours of the US and MRI frames utilizing our automatic algorithms. The constructed neural network is stimulated by data derived from the US tongue contours, and parameters obtained from the MRI tongue contours are assigned to the output of the system. By varying the number of input parameters, we create several system settings and test the operation and efficiency of the network for all configurations.