Syllable-level Korean Fingerspelling Recognition from a Video

Fingerspelling is a communication system that expresses each character with a series of hand gestures. The system is important for deaf people because it can indicate new nouns which cannot be expressed in sign language. Since the expression style of the system is based on the existing writing system, understanding unique characteristics of each writing system is important to recognize fingerspelling properly. In this paper, we propose a syllable-level fingerspelling recognition framework that utilizes characteristics of Hangul. Unlike the conventional frameworks that recognize fin-gerspelling in the phoneme-level, the proposed framework adopts a Hangul-specific attention mechanism that pays more attention to important video frames in terms of recognizing each syllable. In our experiments on the KETI Korean finger-spelling dataset, the proposed framework exhibits more accurate results than a phoneme-level recognition framework.