Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme

An algorithm for speaker's lip contour extraction is presented in this paper. A color video sequence of the speaker's face is acquired under natural lighting conditions and without any particular make-up. First, a logarithmic color transform is performed from RGB to HI (hue, intensity) color space. A Bayesian approach segments the mouth area using Markov random field modelling. Motion is combined with red hue lip information into a spatiotemporal neighbourhood. Simultaneously, a region of interest and relevant boundary points are automatically extracted. Next, an active contour using spatially varying coefficients is initialised with the results of the preprocessing stage. Finally, an accurate lip shape with inner and outer borders is obtained with good quality results in this challenging situation.