Cross-Cultural Facial Expression Recognition: A Deep Learning Study On Indian Cultural Diversity

Facial expression recognition (FER) is essential for human-computer interaction, psychology, and other emotion-based applications. The cross-cultural variation in facial expressions adds a new dimension to learning the behaviour and emotional expressions of different cultures. However, the literature encompassing cross-cultural for FER is sparse. India alone can be divided into five main regions due to wide racial, ethnic, and geographic diversity. Hence, this work presents the first cross-culture facial expression recognition dataset (ICC-FER) focused on the five Indian regions and their rich cultural diversity. The dataset is collected in the wild with a proposed multilevel annotation technique. Subsequently, a deep network is customized for robust feature extraction and classification. It compares seven benchmark facial expressions across different cultures. With this, a comprehensive analysis has been performed by categorizing facial expressions into i) separate regions, ii) expressions without cultural information, iii) classification of the cultures, and iv) integrative analysis and classification of emotional expressions across cultures. Various activation techniques are explored to visualize intermediate learning of the network and highlight the attentive regions of the faces for the specific expression. In addition, the correlation between the emotions expressed across the different cultures has been analyzed using the t-SNE technique. It is observed that synergizing cultural information along with expression provides a significant boost in expression recognition.