Multi-model Human-Computer Interaction System with Hand Gesture and Eye Gesture Control

This research paper study, a new virtual mouse system that combines hand and eye gestures to produce a non-contact interface for smooth computer interaction is designed and put into practice. Accurate cursor movements on the computer screen are produced by the system through exact detection of hand and face position and orientation. The real-time camera video feed is processed and analyzed using OpenCV, a powerful open-source computer vision toolkit, to identify these movements as dynamic pointer operations. One of the system’s key features is its click capabilities, which enables users to give orders with a combination of hand and eye movements. Interestingly, the device incorporates eye-tracking functionality, adding a new level of interactivity to improve user engagement. The device tracks the user’s eye movements to control the cursor, providing a hands-free option for those with limited hand mobility. A volume control function that is triggered by hand gestures is introduced by the system to address audio control. Users can effortlessly and successfully control the audio output by dynamically varying the computer’s audio volume based on the separation between fingers. The user experience is extensive and engaging due to the unique combination of audio control, eye tracking, and hand gestures.