Development of automatic video editing system based on stereo-based head tracking for multiparty conversations

This paper presents an automatic video editing system based on head tracking for multiparty conversations. Archiving meetings is attracting considerable interest. Conventional systems use a fixed-viewpoint camera and simple camera selection based on participants' utterances. However, conventional systems fail to adequately convey to the viewers who is talking to whom. We focus on the participants' head orientation since this information is useful in detecting the speaker and who the speaker is talking to. In order to automatically estimate each participant's head orientation, our system combines modules for stereo-based head tracking. The system selects the shot of the participant that most participants are looking at, based on majority decision. Experiments confirm the effectiveness of our system in several 3-participant conversations.