Hand Gesture Recognition for Patient Monitoring in the Medical Field: A Deep Convolution Neural Networks Approach

Gesture recognition enables computers to understand the language communicated through human body movements. Hand Gesture Recognition (HGR) has emerged as a technology with substantial potential for diverse applications. Hand gestures hold significant value in computer interaction, given their primary role as expressive forms of human communication. This paper aims to demonstrate the viability of HGR systems in the medical domain, specifically for patient monitoring purposes. The work aims to create a monitoring system that utilizes patients’ hand gestures to send urgent messages to an application, ensuring prompt notification of doctors about the patient’s condition. To achieve Hand Gesture Recognition, the researchers utilize a trained Convolutional Neural Network (CNN) model within the MediaPipe framework, coupled with the open cv library, for detecting and identifying hand gestures captured by the camera. The processed data is then relayed to the application via a Raspberry Pi, enabling the transmission of appropriate messages based on the recognized hand gesture. The MQTT (Message Queueing Telemetry Transport) protocol is adopted as the means of communication over the internet, ensuring efficient message exchange between the monitoring system and the application. The obtained results are meticulously analyzed, enabling the calculation of system accuracy. The primary objective of this research is to verify the practicality and effectiveness of incorporating Hand Gesture Recognition (HGR) systems into medical settings, with the overarching goal of improving patient monitoring and care standards.