Voice and Hand Gesture-Based Controlled Computer System

We propose a new Human-Computer Interaction (HCI) system that integrates both hand gesture recognition and voice commands, merging these modes of interaction to enhance accessibility and usability. This innovative approach addresses the limitations of traditional input devices, such as keyboards and mice, which can pose challenges for people with disabilities or those requiring hands-free control. Our system utilizes Python, OpenCV, MediaPipe, PyAutoGUI, and Speech Recognition to replicate the functions of physical devices, enabling users to perform tasks such as file navigation, application control, and typing without the need for conventional hardware. Our project effectively translates hand gestures and voice commands into system operations by offering a user-centered, non-invasive control solution. Furthermore, comprehensive usability testing will be conducted with a focus on individuals with diverse disabilities to refine and validate the system’s accessibility benefits. Aligned with Sustainable Development Goal (SDG) 10, this project contributes to reducing inequalities by providing an inclusive technological gateway for individuals with disabilities, promoting social innovation, and bridging the technology divide. This system not only supports those with disabilities but also broadens hands-free operational possibilities across various settings.