Integrated person identification using voice and facial features

Real-world automatic person recognition requires a consistently high recognition accuracy which is difficult to attain using a single recognition modality. This paper addresses the issue of person identification accuracy resulting from the combination of voice and outer lip-margin features. An assessment of feature fusion-based on audio-visual feature vector concatenation, principal component analysis, and linear discriminant analysis-is conducted. The paper shows that outer lip margins carry speaker identity cues. It is also shown that the joint use of voice and lip-margin features is equivalent to an effective increase in signal-to-noise ratio of the audio signal. Simple audio-visual feature vector concatenation is shown to be an effective method for feature combination, and linear discriminant analysis is shown to possess the capability of packing discriminating audio-visual information into fewer coefficients than principal component analysis.