Hand Gesture Tracking and Recognition based Human-Computer Interaction System and Its Applications

In this paper, we propose a human-computer interaction (HCI) system for entertainment or education via a depth-sensing camera. The whole system is comprised of three modules: hand detection, hand tracking, and gesture recognition. In our system, specifically, hand detection is based entirely on computer vision and do not use any markers. We utilize the Kalman filter and depth data from the Kinect to predict the hand position making the tracking smooth and robust. And we extract the apparent gestural features to recognize gestures, which are adaptable and quick. Two applications are in place to demonstrate the system: writing board, in which the user can freely draw lines in the air using their hand, and a classic game, Gluttonous snake, which is transformed into a novel game by using our framework. The system we develop is highly extendible and can be used in a wide variety of other interactive scenarios.