Attentive Fusion for Efficient Wrist-Worn Gesture Recognition Based on Dual-View Cameras

Wearable hand gesture recognition has attracted considerable attention in the field of smart sensors, human–computer interaction, etc. In this article, we propose a portable device for gesture recognition based on dual-view wrist-worn cameras. Our device is easy to wear and will not impede hand movement. The main challenges we face are the sever occlusion and diverse lighting conditions, which significantly hinder the accuracy of gesture recognition. To mitigate these issues, we propose a novel framework based on attentive feature fusion. The overall framework mainly consists of three components, including an adaptive feature amplification module, an attentive feature enhancing module, and a cross-view attention module. Our intuition is that the detailed features from palm and dorsal hand regions can complement each other and can be regarded as strong cues for inferring hand gestures. Moreover, by adaptively adjusting the brightness of the images, the proposed method can be adaptable to diverse lighting conditions. Our method is efficient and quite effective. By comparing our work with other state-of-the-art methods, we achieve superior performance under various experiment configurations. Extensive experimental results validate that our proposed framework surpasses existing state-of-the-art works by a significant margin.