A Vision-Based Hygiene Monitoring System: Using Deep Learning to Assess Handwashing Procedure in Real Time

In a healthcare environment, workers must follow proper handwashing protocol to prevent the transmission of pathogens. The World Health Organization (WHO) recommends six handwashing motions to ensure that all parts of the hands are cleaned properly. This project aimed to apply deep learning to identify whether a subject is correctly performing handwashing motions in compliance with the WHO protocol. Video footage from ten subjects was used to train a ResNet-18 neural network capable of identifying the six different hand poses. All training was done using the Amazon Web Service (AWS) cloud platform. The resulting model was deployed to an AWS DeepLens camera to produce real time evaluations of live handwashing footage. The system consistently identified the correct handwashing motions with an 88.5% average accuracy and displayed the live inferences on a connected monitor instantaneously. The results demonstrated the practicality of using deep learning to develop a low-cost vision-based system for monitoring handwashing technique in real time.