Networked gesture tracking system based on immersive real-time interaction

Gesture as a natural and efficient interactive mode, which has been widely used in the field of human-computer collaboration, as the present existing gesture acquisition method is difficult to meet the users' immersion experience and ensure the real-time requirements, in this paper, we design a wearable interactive system which can meet the need of real-time hand gesture acquisition and 3D display. From the perspective of human ergonomics, we analysis the relationship between the movements of bones and joints during hand movement and establish a dynamic model about the skeletal structure of hand. On the basis of this theory, combining with the spatial navigation theory and data fusion method of heterogeneous sensors, a hand tree sensor network based on MEMS inertial sensor is established to realize the real-time tracking of gesture. At the same time, we make a comparison and verification of the gesture data by combining with the image processing method through extracting the key frame information in the gesture video. Finally, we can find the system established in this paper can realize the real-time tracking of gestures through analysis and comparison of real gesture, which provides certain reference value.