Hand Gesture-Based Character Recognition Using OpenCV and Deep Learning

Fast, accurate, and user-friendly human-computer interaction (HCI) requires both processing and intelligence. Understanding signs, and symbols is already possible by computers but recognizing symbols drawn live by a human in front of a camera is still a new concept. Many attempts have been made to achieve this already by using different sensors like Time of Flight (ToF) camera, Kinect sensor, etc., by using special metric systems or by special algorithms. Our research proposes doing such work using normal cameras that almost every computer has already. In this work, we tried to approach the problem from two directions. We left the detection, tracking and drawing tasks on mathematics-based algorithms like Accumulated Weight, CSRT (The Channel and Spatial Reliability) Tracker and OpenCV (Open Computer Vision) library. The recognition relies on deep learning. Our model can classify different characters of English alphabet and numerals so that when a user draws that, it can predict that. Our deep learning model is 98.56% accurate in classifying symbols which is more accurate than previous methods while not requiring any special sensors.