Improved Action Unit Detection Based on a Hybrid Model

Facial action detection and facial expression recognition are two closely intertwined problems in behavior analysis. This paper presents evidence that model architectures designed for facial expression recognition can be seamlessly adapted for the action units detection task, taking advantage of the structural similarity between the two problems. As a sample case, we have adapted the Pyramid crOss-fuSion TransformER (POSTER) model for action unit detection by adjusting the architecture to handle a multilabel problem with one output per action unit. Then, we tuned the training parameters and retrained the model to achieve state-of-the-art performance on two widely used datasets: DISFA and BP4D. The results obtained with a standard 3-fold cross-validation setup show an average F1 score of 67.8% for DISFA and 65.5% for BP4D. These results outperform state-of-the-art models for AU detection, support the effectiveness of the approach, and suggest placing higher efforts on adapting existing architectures to leverage the synergies between facial expression recognition and action unit detection.