Emotion Recognition from 3D Motion Capture Data using Deep CNNs

Designing computer games requires a player-centered approach. Whilst following guidelines and functional requirement specifications is part of the process, observing and measuring qualities of the players experience is key in providing feedback to game designers. Moreover, it can also be used to create adaptive and personalized experiences for players. With the advancement of affective computing and gaming user interfaces, the opportunity to recognize the player's emotions becomes more feasible and each different modality can offer additional information as affect expression is less defined as compared to action selection. This paper explores the use of 3D skeleton motion data transformed to 2D images that encode pose and movement dynamics to represent annotated emotions. The 2D images are then used to train and test the Inception V3 CNN model on a binary classification emotion recognition between happy and sad emotions. Preliminary results in unseen test data indicate that the above transformation technique can capture emotional information. The paper also discusses future directions that may improve the effectiveness of the proposed method on a wider scale.