A Lightweight Two-stream Spatial Temporal Convolutional Neural Network for Human Actions Recognition

The convolutional operations of convolutional neural networks (CNNs) on images are untargeted, and the high model complexity of CNNs makes them less suitable for deployment on low-performance devices. According to these, a two-stream spatial temporal convolutional neural network based on channel pruning compression and coordinate attention is proposed to recognize human actions in videos. Specifically, both the temporal and spatial stream sub-networks of the two-stream network are equipped with spatial temporal convolutional neural networks. And the coordinate attention module is introduced to improve the spatial temporal convolutional neural network, which improves it’s abilities to concern about the important information. A coordinate-attention based channel pruning scheme for spatial temporal convolutional neural networks is proposed to achieve network compression. And the compressed model has fewer parameters and can be deployed on computing devices with low performance. Experimental results on HMDB51 and UCF101 demonstrate that the presented model can acquire higher recognition accuracy.