Gesture Controlled Mouse Navigation : Hand Landmark Approach

Euclidean Distance has been an important metric for the calculation of distances between points on a scaled space. Recent times have seen an increased use of this metric for the purposes of pattern recognition. Hand Landmarks are a group of twenty-one points which define the skeletal structure of a human’s hand. Calculation of distances between Hand Landmarks using Euclidean Distance can cater to the need of an efficient algorithm which could recognize patterns using mathematical calculations. The last few years have witnessed the introduction of technologies which promoted virtualization of important hardware thereby, increasing the portability and handling of the systems. The Mouse is an important aspect of the computer which helps to easily interfacing with the system. History has seen the use of trackballs, laser sensors and track pads as means of controlling the system however, each one of them consisted of costly hardware equipment be it the material or the sensors used. This paper proposes a virtual mouse system which can be controlled and navigated using gestures made using any one hand, whether left or right, without the involvement of any external hardware or sensor. All this is done by means of Euclidean distances and hand landmarks. It allows the user to control all the functioning of the mouse including its movement, mouse clicks and drag and drop without physically touching any hardware. It uses only the system’s inbuilt camera to carry-out the functioning of the mouse. The system is carried out using Python along with OpenCV, Mediapipe and the PyAutoGui libraries. All the gestures become visible on the camera screen with appropriate marking to facilitate representing the working of the system.