Multimodal Apparent Personality Analysis based on ConvNeXt-GRU and Random Forest

With the integrative development of computer science and psychology, apparent personality analysis has gradually become a new research hotspot. In this study, we aim to extract key information from large-scale video data and build an end-to-end deep learning model to explore the latent personality traits of vloggers in short videos. To fully utilize the spatiotemporal information existing in audio and visual modalities of short videos, we propose a multimodal apparent personality analysis method that integrates neural network and random forest. In visual modality analysis, we improve the ConvNeXt network by introducing the ECA attention mechanism and propose a new network structure unit named ECblock in order to obtain features closely relating to personality traits. Considering the temporal characteristics of short video, we combine the improved ConvNeXt with gated recurrent units for making personality traits assessment. For the audio modality, we manually extract voice features using librosa tool and employ random forest model for feature prediction under the help of grid search. Finally, according to decision fusion strategy, the latent information learned from visual and audio modalities are fused to implement apparent personality analysis. The results of comparative experiment on ChaLearn First Impressions V2 dataset show that our proposed method achieves competitive performance in terms of prediction accuracy.