3D Hand Gesture Coding for Sign Language Learning

Hand gesture and sign language are important in facilitating non-verbal communication between human beings. This paper proposes a new approach to code 3d hand gestures for presentation and recognition. The coding is based on 3D hand and finger structure extracted from Leap Motion Controller (LMC). From the 3D coordinates tracked by device, shapes, positions or distances between finger tips are coded and normalized according to a set of rules. This coding process can be completed manually or semi-automatically. The recognition stage is automatic. In experiments, our system applied automatic coding and recognition on a subset of Chinese sign language, which is a special set of meaningful hand gestures. Integrated in a Unity application, the method can recognize learned gestures quickly and accurately. For unknown gestures, the system also provided interactive learning method to further extend recognizable alphabet signs.