Optimal video handling in on-line hand gesture recognition using Deep Neural Networks

Deep Neural Networks (DNNs) are machine learning models with a myriad of uses, such as enabling tools that support professional workers or facilitating new modes of human-computer communication. Hand gesture recognition from RGB video feed is one of the most important relevant tasks where DNN models tend to excel, with many application domains typically demanding their real-time execution (e.g., in human-robot interaction scenarios). However, several operational parameters of similar models, especially concerning the way the input video data are handled, have not yet been sufficiently investigated with a clear goal of identifying best practices. For instance, various design choices about how video frames are fed to the DNN model during its training and its evaluation (e.g., using the traditional temporal subsampling per video approach, or employing a temporal sliding window) directly affect method accuracy. This paper aims at empirically finding optimal strategies regarding such operational parameters for input video handling when training and evaluating learning models that perform real-time, on-line gesture recognition’ using typical CNN-based approaches on a relevant dataset.