Two features combination with gated recurrent unit for visual speech recognition

Recently, deep learning has attracted in the visual speech recognition (VSR) field, however, most of approaches use single feature. This paper proposes two features; image-based feature by autoencoder, and motion-based feature based on feature points, and combination approach with gated recurrent unit for VSR task. The proposed method was evaluated on a public dataset Oulu VS, and the speaker independent setting was carried out. Compared with the state-of-the-art, our method obtained the highest accuracy.