Dual IMU-Based Hand Gesture Recognition With Time–Frequency Feature Fusion

With the rapid progress of some novel sensing techniques and artificial intelligence, the miniature inertial measurement unit (IMU) enabled wearable hand gesture recognition (HGR) has become a hotspot research topic. The main challenges facing this area are the accurate segmentation of continuous gestures and the feature-based classification that could potentially improve the recognition accuracy. This investigation presents a dual-IMU sensor nodes-based dynamic HGR system with time–frequency feature fusion powered classification. First, an extended Kalman filter (EKF)-based dual-node data correction method is developed to reduce body movement interference through the data fusion of the two IMUs. Second, an adaptive dynamic gesture segmentation method combining multilevel thresholds zero-speed detection is proposed, and a majority voting mechanism is introduced to determine the gesture states and to eventually improve the segmentation accuracy. Then, a time–frequency feature fusion network is established to enhance the feature representation by converting 1-D time series data into Gramian angular field (GAF) images for 2-D feature extraction, followed by feature fusion and classification to enhance the recognition accuracy. In order to verify the effectiveness of the methods proposed in this study, a dataset containing 26 uppercase English letters air-writing gestures is constructed. Experimental results have demonstrated that the proposed method can effectively realize dynamic gesture segmentation and recognition with the highest recognition accuracy of 99.31%, and the average accuracy of 30 classification experiments reached 97.64%. The presented techniques and methods provide valuable references solutions for wearable HGR.