Action Recognition in Still Images Based on Cross-Modality and Knowledge Distillation

Action recognition in still images is a challenging task compared to video action recognition due to the limited information provided by a single image. Typically, many approaches for action recognition in still images involve two parts: feature extraction and classification. This paper follows the basic paradigm and proposes a Text-Image Alignment(TIA) method that utilizes a text-image alignment classifier based on ground truth labels to assist recognition. Besides, TIA adopts various prompt templates to improve the generalization ability of the TIA classifier. Additionally, this paper proposes Knowledge Distilllation(KD) training strategy by introducing a pre-trained image classifier that serves as a teacher classifier to improve the performance of TIA classifier. And the way to optimize the performance is to minimize the distance between two classifiers. As a result, a novel method is proposed in this paper. To validate the generalization and effectiveness of TIA-KD, experiments are conducted on two different basic archi-tectures: Transformer and CNN on Stanford40, PascalVOC, and HICO, respectively. The experiment results show that the proposed method outperforms baselines by 1~4.5% on above three datasets, respectively.