Emotion Inference of Game Users with Heart Rate Wristbands and Artificial Neural Networks

This paper proposes to infer emotion of game users with a platform that integrates heart rate wristbands, smartphones and emotion inference mechanisms on backend computers. The inferred user emotions can subsequently be utilized by game developers to promptly understand the user reaction to the game content in order to improve the game quality. In this paper, the subjects play four kinds of games that can stimulate emotions such as Pleasure, Happy, Fear and Angry. The subjects' heart rate signals and facial expressions are recorded when they play the games. The facial expressions are then used to determine the emotions of the subjects as the ground truths. The heart rate signals with determined emotions are subsequently extracted and used to calculate the normalized features. Seven normalized features and determined emotions are then used as inputs and outputs to Artificial Neural Network (ANN) for machine learning. The trained ANN model can subsequently be used to classify the emotions of the subjects when they play games. Experimental results show that the average recognition accuracy of the players' emotions of Pleasure, Happy, Fear, Angry and Neutral are 84.41%, 79.08%, 86.81%, 88.64% and 75.30% respectively. We have also selected two, three and four out of the five emotions for recognition as comparison, and the results show that inclusion of more emotions will lead to less recognition accuracy. In the end, data of all the subjects have been merged into a big data set for training and testing. The overall recognition rate falls maybe because of the uniqueness of each subject. It is thus recommended that personal data of a subject should be used as training data for better recognition results.