Utilizing YOLO for Efficient Indonesian Sign Language Recognition

As digital cameras become more prevalent in today’s technology, more and more people are reaping their benefits. A tiny group of people, the disabled community, particularly the Deaf and the Speech Impaired, do not benefit as much or even at all. The purpose of this proposed system is to help those who want to converse with the Deaf and Mute through sign language. The system is developed using YOLOv5 algorithm with YOLOv5s as its pre-trained model. The YOLOv5s model will then be used to train new sets of data with brand new configurations. The newly trained model will be used to classify 26 letters of the Indonesian Sign Language alphabets. To test the efficiency of the system, a few scenarios will be performed such as the distance of the camera, background wallpaper, and the area’s luminosity level. The output of this research is the model can detect 26 letters of the Indonesian Sign Language alphabets on real-time without getting affected by the area’s background and luminosity level but the distance between camera and object does hinder the model’s ability to detect the sign languages. The model runs best on training scenario of 70%:20%:10% dataset distribution with 300 epochs, 16 batch size and 0.01 learning rate which results on 99.27% mAP@0.5 value.