Lip-Sync: the Evaluation of Audio-to-Video Timing Errors over Shot Intervals

The relative timing of sound and video components of the television signals is an important point of the viewers' perception of television programs. There is a complicated problem of a local time-base error at the film dubbing. It needs to measure how well the pronounced words and audio fit the actor's face. The evaluation of local audio-to-video timing errors over short time intervals is the main goal of current paper. This approach is based on representation of speech as a timed sequence of phonemes, prediction of corresponding visemes and measuring the degree of correlation between real and predicted visemes