A Top-View Hand Gesture Recognition System for IoT Applications

Gesture recognition integrated into Internet of Things (IoT) systems is a growing field with opportunities for improvement. Identifying hand gestures from vertical angles with a cooperative network of cameras can enable a broader collection of control methods for controlling IoT devices within residential and commercial environments. Integrating this smart home framework with other IoT control methodologies also creates a more user-friendly experience. To generate a proof of concept of the effectiveness with this control framework, a convolutional neural network was created using the YOLOv7 deep learning architecture and Roboflow software. This image detection network was connected to a simulated IoT device via Home Assistant home automation software and communicates using the MQTT protocol. The gesture detection system was tested on different camera setups to evaluate the importance of camera position and data sharing within a smart-home device network.