Mid-Air Gestures for Multi-Fingered Virtual Assembly System with Leap Motion

With the development of human-computer interaction and virtual reality technology, virtual assembly shows its advantages in the fields of robotic and mechanical assembly. However, the traditional virtual assembly is based on the mouse and keyboard devices to interact with the virtual environment with low interactivity and poor sense of reality. This paper proposed a novel approach for designing and implementing a virtual assembly training system based on multi-fingered mid-air gestures with Leap Motion controller. The procedures included the construction of assembly scene, the functional identification of the mid-air gestures and the design of the assembly process. This framework is validated through a sample assembly process of a single-stage gear reducer. Experimental results indicated that the system works well to implement assembly operations with immersive experience and high gesture recognition accuracy. From the results, it can be seen that the proposed system has the potential to be a useful tool for mechanical assembly education and industrial design and manufacturing.