LM-Net-Dual: A Two Branch Spatiotemporal Feature Extraction Model for Dynamic Gestures Recognition

Dynamic gesture recognition is one of the important technologies in human-computer interaction. Leap Motion Controller can capture the coordinate data of key points of the hand, and is widely used in Gesture recognition. Current works usually rely on manual feature extraction for LMC-based gesture recognition. The purpose of this paper is to propose an end-to-end dynamic gesture recognition scheme in real time. A spatiotemporal data recoding method is proposed, which recodes the original data containing the spatial and temporary features returned by LMC, so that 2D convolution can extract features from space and temporary dimensions. The proposed two branch model LM-Net-Dual based on task splitting can focus on finger posture and hand movement separately, improving the discrimination of similar gestures. In addition, a real-time dynamic gesture recognition scheme based on sliding time window and confidence limit is proposed. The proposed model is trained and tested on dataset LM-16 which contains 16 dynamic gestures, achieving an accuracy of 100%.