Data Fusion for Geometrical and Pixel Based Lip Feature

Lipreading is applied to synthesize speech for the speech-impaired people. To get a higher recognition result, data fusion with weighting coefficients at feature level is used to integrate the lip information from different kinds of lip features. Experiments are carried out based on HMM with different states and Gaussian mixture component in a small database for speaker-dependent case. Experiment results showed that the integrated discriminate vector after feature fusion obtains the information from the Geometrical feature vector of lip region and the DCT coefficients of lip' ROI. With best weighting coefficients m: n=1.5:1, the recognition rate are improved by as much as 5.02% and 8.37%, respectively.