By Topic

Computer lipreading for improved accuracy in automatic speech recognition

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
P. L. Silsbee ; Dept. of Electr. & Comput. Eng., Texas Univ., Austin, TX, USA ; A. C. Bovik

Among the various methods that have been proposed to improve the robustness and accuracy of automatic speech recognition (ASR) systems, lipreading has received little attention until very recently. However, results from the psychological literature indicate that lipreading, in conjunction with auditory perception, can provide a strong improvement in speech recognition and understanding in humans. We have developed a novel speaker-dependent lipreading system that uses hidden Markov models. An audiovisual system known as Lipreading to Enhance Automatic Perception of Speech (LEAPS) is described, in which the lipreading system is used in conjunction with an audio ASR system in order to improve the accuracy of the latter, especially under degraded acoustical conditions. Experimental results are presented for two small phoneme discrimination tasks, as well as a medium vocabulary isolated word recognition task. In all cases, performance of the combined system is superior to that of the audio system, with a reduction in errors ranging from 20 to 65%

Published in:

IEEE Transactions on Speech and Audio Processing  (Volume:4 ,  Issue: 5 )