A Skeleton-based Dynamic Hand Gesture Recognition for Home Appliance Control System

In recent years, advances in 3D sensors have dramatically promoted the development of dynamic hand gesture recognition research. On the other side, the task of hand pose estimation has seen significant progress due to the powerful feature extraction capabilities based on Convolutional Neural Networks (CNNs). In this paper, we present a lightweight CNNs method on hand gesture recognition for home appliance control system. We propose a two-stage CNN model to facilitate it. At the first stage, we utilize DetNet to detect the hand and generate 3D hand skeleton locations. At the second stage, a skeleton-based dynamic hand gesture recognition model is developed. We have 99.4% accuracy by the trained CNN model with the testing dataset. Besides, we implement this system on the Nvidia Jetson AGX Xavier to control the on/off of the fan and the light.