The prompt of lip shape modification of cacology based on the speech evaluation techniques — a case of basic Chinese learning

In the study, a Chinese learning assisted system based on speech recognition and lip shape image processing is proposed. The mel-frequency cepstral coefficient (MFCC), the pitch contour, and energy curve were adopted as the parameters of voiceprint, speech tone, and magnitude of speech signals, respectively. On the other hand, the height and width of the lip shape were sent into the lip shape analysis. In the scoring stage of speech utterances, the dynamic time warping (DTW) algorithm and probabilistic neural network (PNN) were applied to determine whether the test speech was qualified or not during Chinese learning process. The simulation results indicated that the hybrid of MFCC, pitch contour, and energy curve parameters of speech signal could slightly promote the accuracy of classification—could achieve up to 90%. Finally, the Receiver Operating Characteristic Curve (ROC) was introduced to quantitatively evaluate the sensitivity and specificity of the performance of the proposed algorithm.