Facial Expression Recognition with Face Mask using Attention Mechanism and Metric Loss

Facial expression recognition has been widely studied and applied in various fields such as healthcare and commerce. However, with the onset of the COVID-19 pandemic, people around the world started wearing masks. Unfortunately, there have been relatively few studies focusing on facial expression recognition under mask-wearing conditions. Therefore, this paper proposes the Masked Facial Expression Classification Network (MFN). Firstly, MFN extracts global features from input images as well as captures local features to mitigate the negative effects of unnecessary information, such as masks, and to emphasize crucial facial regions like the eyes. Secondly, incorporating an attention mechanism into the feature extractor, it allows the model to focus on important regions and channels for facial expression recognition. Lastly, MFN incorporates the metric loss as a discriminative loss because there is a tendency for high similarity in feature distributions among each class for facial expression recognition. This loss aims to minimize the distance between feature vectors of the same class and maximize the distance between feature vectors of different classes. In the experiment using the dataset with individuals wearing masks, the proposed network demonstrated improved accuracy compared to other models.