AMV-TSN: Temporal Segment Networks Based on Appearance and Motion-variation for Videos

With the growing demand for video content analysis, action recognition has attracted lots of interest. Similar actions in videos differ in speed, direction, acceleration and spatial appearance. However, existing methods do not fully explore the appearance and motion information which is vital for action recognition. Therefore, we propose a novel temporal segment network based on appearance and motion-variation information (AMV-TSN). Our contribution is two-fold. First, a novel method is proposed to extract the appearance and motion-variation information of videos. Second, The AMV is design to obtain more valuable information of actions. Experiments demonstrate that the temporal segment network based on more effective AMV achieves better performance on action recognition. We carry out experiments on two popular datasets: UCF101 and HMDB51. Experimental results indicate that AMV contains more distinctive motion-variation features and our model outperforms previous works.