Hand Gesture Recognition For Tone Classification According To Kodaly Handsign Using CNN

In recent years, hand gesture recognition has received attention for its wide range of applications and its ability to efficiently interact with machines. Hand signs have become part of basic music theory in educational settings, one of which is through the use of Kodaly hand signs. Kodaly Hand Signs are used as a concept in interactive angklung performances to determine the notes to be played. Thus, the automation of hand gesture recognition in playing music is one of the innovations in the field of artificial intelligence and machine learning. This paper discusses a deep learning convolutional neural network model that can classify 8 Kodaly hand signs in real-time. The hand gesture recognition process begins with image segmentation, displays the gray intensity value, resizes the image, then proceeds to the preprocessing stage by converting the image into a vector form, then the image data is trained to produce a model. It is models such as these that will be useful in education in providing a human-computer interface for musical expression. The experimental results show that the proposed CNN model achieves an average accuracy rate of 98.29%, 99.95% precision, 99.97% recall, and 99.96% F1 score.