Static and Dynamic Human Arm/Hand Gesture Capturing and Recognition via Multiinformation Fusion of Flexible Strain Sensors

Human arm/hand gestures capturing and recognition provide an intelligent and convenient way for use in application from human-machine interface (HMI) and human-computer interaction (HCI) to human-robot interaction (HRI). As human gestures constitute a powerful inter-human communication modality, they can be considered as well an intuitive and convenient mean for the communication between human and machines. This paper presents a novel real-time static and dynamic human gesture capturing and recognition method designed using flexible wearable data band/data glove, which are designed based on three/ten stretchable strain sensors respectively. The wearable data bands and data gloves are worn by human arm/hand to ensure that the sensors are accurately attached to the human joints for accurate measurement of joints movements of shoulders joints, elbows joints, wrists joints, metacarpal and proximal joints of fingers. In this work, a new idea of real-time static and dynamic human gesture capturing and recognition is introduced and developed based on the radial basis function neural network (RBFNN). Dynamic time warping (DTW) is used to select dynamic behavior candidates and also to recognize gestures by comparing the observed records with a series of pre-recorded reference data patterns. The solution deals simultaneously with static and dynamic gestures as well as with multiple joints within the interest space. The experimental results of human arm/hand static gestures and dynamic gestures capturing and recognition verify the effectiveness of the proposed methods.