Personality Assessment Based on Multimodal Attention Network Learning With Category-Based Mean Square Error

Personality analysis is widely used in occupational aptitude tests and entrance psychological tests. However, answering hundreds of questions at once seems to be a burden. Inspired by personality psychology, we propose a multimodal attention network with Category-based mean square error (CBMSE) for personality assessment. With this method, we can obtain information about one’s behaviour from his or her daily videos, including his or her gaze distribution, speech features, and facial expression changes, to accurately determine personality traits. In particular, we propose a new approach to implementing an attention mechanism based on the facial Region of No Interest (RoNI), which can achieve higher accuracy and reduce the number of network parameters. Simultaneously, we use CBMSE, a loss function with a higher penalty for the fuzzy boundary in personality assessment, to help the network distinguish boundary data. After effective data fusion, this method achieves an average prediction accuracy of 92.07%, which is higher than any other state-of-the-art model on the dataset of the ChaLearn Looking at People challenge in association with ECCV 2016.