Research on an automated speech pattern recognition system based on lip movement

To develop a communication support system for voice-impaired people, the authors use a simple image processing device to input visual lip movement information. Unlike precious lipreading research, the authors have not focussed their work on correlating meagre lip movement information with the verbal artefacts they are used to produce, rather they look at the lip movement information as a special language in its own right. Imitating the process of verbal language comprehension in the human brain, the authors first cut the visual information of a sentence into segments and encoding them by comparison with a set of standard patterns established by a self-organizing feature map neural network. Next, the authors recognize the sentence by matching its code sequence with that of the sentences in the recognition task. With this approach, the authors achieved a recognition rate of 80% in a twenty Japanese sentence recognition task.