A robust hierarchical lip tracking approach for lipreading and audio visual speech recognition
Lei Xie
Xiu-Li Cai
Zhong-Hua Fu
Rong-Chun Zhao
Dong-Mei Jiang
Sch. of Comput. Sci., Northwestern Polytech Univ., Xi'an, China;
Abstract
This paper presents a robust hierarchical lip tracking approach (RoHiLTA) for lip-reading and audio visual speech recognition (AVSR) applications. Lip regions of interest are subtly detected by motion and facial structure information. Improvements are made on active shape models (ASMs) for extracting lip contours more accurately and efficiently from video sequences of a speaker's talking face in natural lighting conditions and without particular make-ups. Local and global ASM search algorithms are both improved by introducing color information, 2D mouth corner match, and robust estimation. For noise-free features, localization errors are automatically corrected by an interpolating scheme. A fast implementation of the hierarchical approach is also proposed. Extensive experiments show that the improved ASM can effectively reduce the lip locating errors. The fast implementation of RoHiLTA can consistently achieve superior performance to conventional ASMs in lip tracking tasks, and then can be effectively integrated in lip-reading and AVSR systems.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
