Gesture and Voice Controlled Virtual Mouse for Elderly People

Elderly people face unique challenges when using conventional computer interfaces. Therefore, there is an essential need to model a system for such people for easy accessing of modern computer technologies. This paper presents an inventive solution, the "Gesture and Voice Controlled Virtual Mouse" designed to improve the digital interaction experience for older individuals facing challenges with traditional computer input peripherals. Employing advanced technology, this project establishes an intuitive interface using natural gestures and vocal commands. Gesture recognition employs cutting-edge computer vision and machine learning models, for accurate interpretation of automatic hand gestures extraction which is implemented using MediaPipe framework on top of pybind11 along with OpenCV. Voice recognition integrates automatic speech recognition and natural language processing to understand spoken commands effectively. The speech assistant Proton recognizes the commands through pyttsx3 library of python and perform the action required. The experimentation results demonstrated the efficacy of the proposed model in accurate prediction of hand gestures and voice commands.