Studying the Occluded: Hand-Over-Face Gestures in Human Avatars

Hand-over-face gestures contain rich information about complex mental states [7] and have a great potential to support automatic systems of emotion processing [10]. Work by Mahmoud et al. (2016) [33] has provided the first systematic, categorical taxonomy of hand-over-face gestures on a naturalistic complex mental state dataset. Currently, there is no comprehensive dataset of hand-over-face gestures and no systematic study of hand-over-face gesture perception. The purpose of my PhD research project is to build on this taxonomy [33] to create the first synthetic, validated dataset of hand-over-face gestures and to conduct the first systematic study of hand-over-face gesture perception in human participants. To do this, I have incorporated hand-over-face gestures and facial expressions in human avatars and have developed a specific methodology for conducting the first hand-over-face gesture perception study. These will be included in a pilot experiment to get preliminary data whether adding hand-over-face gestures influences ratings of anxious/worried facial expressions in human realistic avatars.