Investigating back propagation neural network for lip reading

Visual Speech processing is the key concern of researchers working in the field of speech processing and computer vision. Though earlier audio speech processing was popularly used for speech recognition but their performance deteriorated in the presence of noise. Moreover, variation in accent is another challenge that affects the performance of such systems. In the presented paper, we explore variation of lip texture features for recognition of visemes. The variations in temporal behavior of lip texture features is coded using Local Binary Pattern features in three orthogonal planes. The classification is carried out using the back propagation neural network, which is a network with hidden layer. The added advantage of hidden layer for lip reading is that it takes into account the nonlinear variation of lip features while speaking. The proposed approach is used for Hindi word recognition and achieved high accuracy at the cost of computation time.