Interactive Educational Game Development Using Modified Single Stage Detector

This research develops a real-time hand gesture detection system using the You Only Look Once (YOLO) algorithm for simple math learning applications aimed at children with special needs. This application is a simple drag and drop game using hand gesture indicators. The system utilizes computer vision and deep learning technology to recognize hand gestures in videos which are then converted into interactive inputs in learning media. System testing involves hyperparameter tuning methods, including learning rate, image size, and momentum adjustments. The test results show that the YOLO algorithm with hyperparameter tuning method with hyperparameter values of learning rate 0.0001, image size 320, and momentum 0.937 is effective in detecting hand gestures with an mAP@0.5: 0. 95 score of 93.27%. The test results provide the potential for using a real-time hand gesture detection system as an interesting and interactive learning method for children with disabilities, especially for those who have limitations in communication.