A robust hierarchical lip tracking approach for lipreading and audio visual speech recognition
Lei Xie; Xiu-Li Cai; Zhong-Hua Fu; Rong-Chun Zhao; Dong-Mei Jiang
Machine Learning and Cybernetics, 2004. Proceedings of 2004 International Conference on
Volume 6, Issue , 26-29 Aug. 2004 Page(s): 3620 - 3624 vol.6
Digital Object Identifier
Summary: This paper presents a robust hierarchical lip tracking approach (RoHiLTA) for lip-reading and audio visual speech recognition (AVSR) applications. Lip regions of interest are subtly detected by motion and facial structure information. Improvements are made on active shape models (ASMs) for extracting lip contours more accurately and efficiently from video sequences of a speaker's talking face in natural lighting conditions and without particular make-ups. Local and global ASM search algorithms are both improved by introducing color information, 2D mouth corner match, and robust estimation. For noise-free features, localization errors are automatically corrected by an interpolating scheme. A fast implementation of the hierarchical approach is also proposed. Extensive experiments show that the improved ASM can effectively reduce the lip locating errors. The fast implementation of RoHiLTA can consistently achieve superior performance to conventional ASMs in lip tracking tasks, and then can be effectively integrated in lip-reading and AVSR systems.
View citation and abstract |
Cart
