Multimodal Apparent Personality Traits Analysis of Short Video using Swin Transformer and Bi-directional Long Short-Term Memory Network

Due to potential applications in online education, human resources and personalized recommendation, apparent personality analysis has become a meaningful task in the field of computer vision and multimedia. According to characteristics of short video in time length, idea density and individual expression, we take full advantage of clues in visual and audio aspects to propose a novel framework of apparent personality analysis. From the viewpoint of visual modality, Swin- Transformer and Bi-directional Long Short-Term Memory (BiLSTM) network are introduced to discovery feature representation for personality assessment. Meanwhile, usual features including chromaticity characteristics, root mean square energy, spectrum, zero crossing rate, and MFCC are extracted to describe personality from the perspective of audio modality. Finally, feature leaned from visual and audio modalities are fused to make personality analysis that can be viewed as solving regression problem. For the purpose of evaluating the proposed algorithm's performance, we conduct comparative analysis between our approach and other methods on First Impression V1 and Short Video Vlogger Personality datasets. The experimental results show that our method is competitive in prediction accuracy.