Human Emotion Estimation through Physiological Data with Neural Networks

Effective collaboration between humans and robots necessitates that the robotic partner can perceive, learn from, and respond to the human’s psycho-physiological conditions. This involves understanding the emotional states of the human collaborator. To explore this, we collected subjective assessments - specifically, feelings of surprise, anxiety, boredom, calmness, and comfort — as well as physiological signals during a dynamic human-robot interaction experiment. The experiment manipulated the robot’s behavior to observe these responses. We gathered data from this non-stationary setting and trained an artificial neural network model to predict human emotion from physiological data. We found that using several subjects’ data to train a general model and then fine-tuning it on the subject of interest performs better than training a model only using the subject of interest data.