Robust joint audio-video localization in video conferencing using reliability information

This paper proposes a new method for performing joint audio-video localization that explores the reliability of the individual localization estimates. The reliability information is estimated from the audio and video data separately. The proposed method uses this reliability information in conjunction with a simple summing voter to dynamically discriminate erroneous outputs from the audio and video localizers while performing fusion on the localization results. Based on the voter output, a majority rule is then used to make the final decision of the active talker's current location. The results show that adding the reliability information during fusion improves localization performance when compared with audio only, video only and joint audio-video using straight summing fusion localization methods. The computational complexity of the proposed method is comparable to the existing ones.