MPRNet: a Temporal-Aware Cross-Modal Encoding Framework for Personality Recognition

Recent advances in personality recognition have improved trait inference from multimodal data, yet many existing methods rely on short-term video segments or static images, limiting the modeling of temporal dynamics due to short video durations, sparse frame-level annotations, and inconsistent modality coverage across audio, text, and visual channels. These limitations make it difficult to model how personality traits manifest over time and across modalities in naturalistic settings. To address these challenges, we introduce the Northeast University Personality Recognition (NEUPR) dataset, comprising 654 self-reported and discussion-based videos collected through MBTI assessments. NEUPR offers naturally expressed multimodal dataincluding audio, facial expressions, eye movements, and speech transcripts-captured across diverse participants and real-world settings. Building on this dataset, we propose MPRNet, a unified framework for dynamic personality recognition featuring two core innovations: (1) a multimodal encoder that leverages LSTM to capture temporal dependencies across longer sequences and integrates latent personality embeddings extracted from BERT representations of text to enrich semantic context, fused through adaptive weighting and enhanced by Gram encoding to preserve local feature patterns; and (2) a feature enhancement module that incorporates learnable positional encoding and channel attention to address modality imbalance and improve sensitivity to spatially salient features across modalities. Experimental results demonstrate that MPRNet outperforms state-of-the-art methods across multiple datasets, while ablation studies confirm the effectiveness of its components. By explicitly modeling temporal variation and enhancing cross-modal fusion, MPRNet enables more robust personality inference. This work establishes both a benchmark dataset and an adaptive modeling framework for multimodal personality analysis, advancing dynamic trait recognition.