Gesture-Based Control of Smart Devices for People with Vocal Disorders using YOLOv7

Using a physical controller for human-computer interaction, such as a mouse or keyboard establishes a strong barrier between the user and the computer. In this research, we developed a hand gesture detection system to effectively monitor dynamic hand gestures. The current methods for recognizing hand gestures are designed by CNN architecture. Real-time recognition of hand gestures from video streams is a challenging task because there is no indication in the video when a gesture begins and ends, performed gestures should only be recognized once, and the entire architecture should be designed with memory and power budgets in mind. In this work, YOLOv7 is a proposed method to address the difficulties above. In terms of speed and accuracy, the suggested YOLOv7 outperformed transformer-based detectors such as the SWINL Cascade-Mask R-CNN R-CNN. YOLOv7 fared better than YOLOX, YOLOR, Scaled-YOLOv4, YOLOv5, DETR, Deformable DETR, DINO-5scale-R50, and VitAdapter-B. After identifying the hand gesture, a corresponding action will be taken based on the recognized hand gesture. In this work, YOLOv7 is compared with YOLOv5 and it is proved that YOLOv7 outperforms YOLOv5 under performance metrics such as precision, recall, and mean average precision.