Speech and Face Biometric for Person Authentication

In this paper, we present a multimodal audio-visual speaker identification system. The proposed system decomposes the information existing in a video stream into two components: speech and lip motion. It has been studied that lip information not only presents speech information but also characteristic information about a person's identity. Fusing this information with speech information will produce robust person identification under adverse condition. Our experiments demonstrate that the visual modality improves person authentication. We investigated the performance of person identification system using three different sets of visual features, one set of speech feature. There will be improvement in the recognition rate of the system when visual and speech features are fused using an adaptive weighing technique.