Action Recognition in Compressed Domain Using Residual Information

Practically, action recognition using deep learning approaches are slow because of high temporal redundancy and large size of the raw video data. One of the solutions for boosting accuracy is calculating optical flows. Generally, extracting motion features are too time-consuming. Therefore, traditional action recognition methods are not suitable for real time applications. On the other hand, compressed videos are available in many situations especially when using mobile devices. We proposed a method that extracts residuals directly from compressed videos by partially decoding the video and feed them to a deep neural network. In general, exploiting the compressed domain features as available information provides a slight reduction in accuracy while the low complexity of this method makes it appropriate for real time applications. The experimental results on multiple first and third person datasets exhibit that while the proposed method provides low computational complexity, the results are highly competitive with traditional approaches in accuracy.