Deep Network-based Hand Gesture Recognition using Optical Flow guided Trajectory Images

The use of body gestures and specially hand ges-tures can be a convenient and useful alternative tool for many utilizations in the human-computer interaction community. A typical hand gesture recognition system comprises different stages like detection, representation and recognition. In this process of hand gesture recognition, proper detection and tracking of the moving hand in a cluttered background play an important role due to the varied shape and size of the hand. In this work, we propose a framework for the recognition of isolated gestures where the moving hand with different shapes, size and colours is detected through optical flow, and the proper hand gesture is recognized using a VGG16 architecture. This paper utilizes the optical flow to track points of interest in video and store the tracked motion as images that we call trajectory-based images. These images are then fed to a VGG16 network for classification. For feature learning and recognition, a deep learning based method is used due to its inherent ability to extract robust and effective features for classification purposes. The main benefits of the proposed method is its simplicity and ease of implementation. This method has offered higher multi-class classification accuracy with a limited amount of continuous isolated hand gesture video dataset.