Joint Selection using Deep Reinforcement Learning for Skeleton-based Activity Recognition

Skeleton-based human activity recognition has attracted lots of attention due to its wide range of applications. Skeleton data includes two or three dimensional coordinates of body joints. Not all of the joints are effective in recognizing an activity. In this paper, we propose a novel framework for identifying relevant joints per activity and using them for the purpose of activity recognition. We propose to formulate the joint selection problem as a Markov Decision Process (MDP) where we employ deep reinforcement learning to find the most informative joints per frame. The proposed joint selection method is a general framework that can be employed to improve the existing human activity classification methods. Experimental results on two benchmark activity recognition data sets using three different classifiers demonstrate the effectiveness of the proposed joint selection method.