Smile When You Read This, Whether You Like It or Not: Conceptual Challenges to Affect Detection

The survey by Calvo and D'Mello presents a useful overview of the progress of and issues in affect detection. They focus on emotion theories that are relevant to Affective Computing (AC) and suggest stronger collaborations between disciplines. My contribution emphasizes the importance of these issues for AC. In fact, empirical research strongly suggests that facial, vocal, and bodily expressions, subjective experience, and physiological changes are often not very highly correlated in spontaneous situations. Overestimating this cohesion limits the usefulness of affect detection methods in real-world applications. Other factors, such as social context, knowledge regarding the goals of certain interactions, as well as interindividual differences are critically important factors for improving affect detection. At times, social concepts, such as politeness, might be more conducive to model realistic behavior. Knowledge on affect perception is important to estimate the level of realism required to create satisfying and productive interactions between users and artificial systems. Interdisciplinary joint research between social and biological scientists on the one hand and computer scientists and engineers on the other is necessary to deal with the complexity of affective processes. All disciplines involved have much to gain in the process.