Action Recognition Based on C3D Network and Adaptive Keyframe Extraction

In this paper, a video action recognition method is developed based on the C3D neural network and the support vector machine. Rather than directly input the considered video clips to the network, an novel adaptive keyframe extraction strategy is proposed to first extract keyframes which are then input to the network. In our keyframe extraction strategy, whether or not the keyframe extraction procedure should be implemented is determined by the extent of information redundancy existed in the video clips and measured in terms of the scene change speed detected based on an RGB histogram. Keyframes are extracted with an adaptive and iterative scheme based on optical flow analysis. With keyframes as input, a C3D convolutional neural network is trained for feature extraction which are then input to a trained support vector machine for behavior recognition. Experimental results show that with the adaptive keyframe extraction scheme involved, the performance of the action recognition method is improved significantly in the UCF101 dataset.