Audio Segmentation and Speaker Localization in Meeting Videos

Segmenting different individuals in a group meeting and their speech is an important first step for various tasks such as meeting transcription, automatic camera panning, multimedia retrieval and monologue detection. In this effort, given a meeting room video, we attempt to segment individual person’s speech and localize them in the video, based on data from a single audio and video source. The segmentation method is driven by audio and enhanced by video cues. We used Bayesian Information Criterion (BIC) to segment the feature vector streams and graph spectral partitioning to cluster them. We compare our results with audio based segmentation method and our localization technique with the commonly used mutual information.