A Human-Object Interaction Behavior Recognition Algorithm Based on Transformer

Human-object interaction behavior recognition is a high-level computer vision task for computers to understand the content of images in real scenes. Thanks to the introduction of Transformer and DETR, the performance of the algorithms for human-object interaction recognition has improved significantly in the first stage, however, the existing models still suffer from inadequate feature extraction and model bias. Based on this, we optimize and improve the algorithm's performance in terms of both fine-grained contextual information extraction and dynamic weighting. We conducted training and testing on the HICO-DET dataset and found through extensive experiments that (1) our proposed fine-grained contextual information-based Human-object interaction behavior recognition algorithm improves the performance of “Full”, “Rare” and “Non-Rare” mAP by 2.48%, 7.42% and 1.44% respectively compared with the original algorithm. (2) During model training, we introduced dynamic weights to improve the optimization of the model for rare Human-object interaction behavior, which can effectively improve the model performance. These experimental results validate the feasibility, effectiveness and adaptability of our proposed fine-grained module, global context extraction module and dynamic weight training strategy.