Online Dynamic Hand Gesture Recognition Using 3D-CNN-RNN Hybrid Architecture

Online dynamic hand gesture recognition is often used in applications that require immediate interaction and response. For example, in virtual reality (VR) or augmented reality (AR) environments, accurate and fast recognition of hand gestures is essential for creating a seamless and immersive user experience. In- car gesture control system allow drivers to interact with infotainment and navigation systems without taking their eyes off the road. Fast and accurate recognition is crucial to maintain driver attention on the task of driving. Thus, the proposed system focus improving its accuracy to the extreme and the efficiency of the model. The proposed system introduces a new model -3D-CNN-RNN for Online Dynamic Hand Gesture Recognition. 3D-CNN is well known for its ability to extract spatial information which is crucial for hand gesture recognition and RNN is well-known for its ability to extract temporal features in videos etc. Hence, 3D-CNN-RNN is powerful model which can produce huge results for this model. After preprocessing the data, the input is fed to 3D-CNN which acts as the feature extractor. The 3D-CNN’s output is a sequence of feature maps that encode the evolving appearance of the hand gesture over time. The RNN component processes the sequence of feature maps produced by the 3D-CNN. The output produced by 3D-CNN is fed to RNN. The RNN component processes the sequence of feature maps produced by the 3D-CNN.The RNN’s sequential nature allows it to model temporal dependencies between frames and capture the dynamic aspects of the gestures.