Temporal Attention Network for Action Proposal

Temporal action proposal, which extracts segments of interests from untrimmed video, is an important step for video analysis. For state-of-the-art temporal action proposal methods, average pooling is often used to aggregate features in deep neural networks, which inevitably ignores the significances of different video clips. Therefore, we propose a Temporal Attention Network (TAN) model to address this issue. Temporal attention with fully connected layers is introduced to adaptively combine clip-level features and form a compact and discriminative video representation. In addition, we show that the learned attention weights could also be used as an effective temporal feature to further improve the performance. Extensive experiments on THUMOS-14 demonstrate that our algorithm performs favorably against the state-of-the-art methods.