GestureNav: Eye and Facial Gesture-Based Cursor Control and Action Triggering

A novel human-computer interaction system using computer vision and machine learning to enable hands-free control of a computer using facial landmarks and gestures. The system integrates and combines OpenCV, MediaPipe, and PyAutoGUI to receive a real-time video feedback from a connected webcam,it works by extracting facial landmarks from the video feed and using this information to control the computer with accurate facial gestures.. The framework also incorporates text-to-speech capabilities using Google Text-to-Speech (gTTS) for auditory feedback and web automation with a web-browser module. Key functionality includes controlling the mouse cursor with eye movements, executing mouse clicks and triggering predefined web actions such as opening websites based on specific facial expressions. The functionality of the system is demonstrated by real-time applications, such as YouTube, Google AI and ChatGPT websites or any preferred website, based on the movement of specific facial landmarks. This new way approach highlights the potential of developing accessible and intuitive interfaces for users with motor impairments, providing a seamless and efficient way to interact with the digital environment.