Multi-speaker voice activity detection using a camera-assisted microphone array

We present a method for voice activity detection of multiple concurrent speakers using a camera-assisted microphone array. The proposed method uses face detection to identify locations of potential speech sources, and uses this information in an adaptive beamforming procedure to form a spatially directed detection algorithm to identify voice activity for individual speakers. Voice activity is classified using support vector machines with mel-frequency cepstrum coefficients as features. To increase the spatial filtering ability of the array we use a combination of Dolph-Chebyshev weighting and null-steering. We have carried out two experiments to gauge the accuracy of the proposed method, and obtain a representative accuracy of around 95% for single speakers, with around 1% loss of accuracy with two simultaneous speakers.