Audio-visual data fusion for tracking the direction of multiple speakers

This paper presents a multi-speakers tracking algorithm using audio-visual data fusion. The audio information is the direction of speakers and the visual information is the direction of detected faces. These observations are used as inputs of the tracking algorithm, which employed the framework of particle filter. For multi-target tracking, we present a flexible data association and data fusion, which can deal with the appearance or absent of any information during tracking process. The experimental results on data collected from a robot platform in a conventional office room confirm a potential application for human-robot interaction.