Gesture-Based Virtual Keyboard: Enhancing Human-Computer Interaction Through Hand Tracking

In this study, an algorithm is developed that utilizes various OpenCV, cvzone and python libraries to provide a virtual keyboard. The program detects hand gestures, specifically the tips of fingers captured through a webcam, to simulate keystrokes corresponding to buttons on the virtual keyboard displayed on the screen, using Hand tracking module provided by Mediapipe. The virtual keyboard consists of keys of ‘QWERTY’ format and are represented by a graphical element. When the index finger approaches the corresponding key, it is highlighted and when distance between the tips of middle finger and index finger matches the required distance the corresponding keystroke is generated. The program captures video frames continuously from the webcam, identifies hand gestures, and accordingly updates the virtual keyboard's display. Additionally, the program outputs the text generated in a designated area of the screen using pynput library. The implementation uses object-oriented programming principles, with classes defined for buttons and hand detection functionality.