Gesture-Driven Virtual Mouse with a Voice Assistant

Hand gesture recognition plays a pivotal role in facilitating human-computer interactions. The system utilises OpenCV to perform real-time hand tracking and gesture recognition, enabling cursor movement control. Along with a novel Voice command detection by the microphone activates various functions, including clicking actions, launching/stopping gesture recognition, Google searching, finding locations on maps, file navigation, and obtaining the current date/time. The system processes the video stream to segment the hand region. It recognises gestures through contour analysis, translating them into mouse controls. By combining hand gesture recognition and speech recognition, the system enables intuitive hands-free control of the mouse cursor and access to additional functions through voice. The hand region is isolated by applying colour segmentation in the HSV (Hue, Saturation, Value) colour space. The contour is detected to track hand shape and gestures. Structural analysis of the contour and convex hull enables gesture recognition. This virtual mouse system enables handsfree computer control, improving accessibility. By eliminating the need for a physical mouse device, it provides a more convenient and intuitive user experience. The proposed virtual mouse system eliminates the necessity for a physical mouse device, allowing hands-free control. This advances human-computer interaction technology by providing a more accessible and convenient interaction method.