Video Tracking Of 2D Face Motion During Speech

We present a video-based system for tracking 2D face motion during speech. The system tracks dot markers on the speaker's face from video sequences. A digital video camera is used to film the speaker during speech production experiments. The acquired digital video is converted into an image sequence, which is then provided as input to the tracking algorithm. The algorithm locates all dots in every frame of the sequence, and outputs the x and y coordinates of the dots in the images over time. Continuity constraints on the face motion are imposed when searching for the dots. The algorithm was tested on video sequences for both spontaneous and non-spontaneous (i.e., reading) speech conditions, and no mistracking occurred for the sequences used