A spatio-temporal neural network applied to visual speech recognition

We present a new neural architecture called spatio-temporal neural network (STNN). In this work, we have utilised the Hermitian distance as the basis of spatio-temporal data comparison to adapt a supervised (RCE) and an unsupervised (K-means) learning algorithms for training the STNN weights. A visual speech recognition (automatic lip-reading) system based on STNN is developed and the results obtained on a French digit recognition task are given.