Revealing Depression Through Social Media via Adaptive Gated Cross-Modal Fusion Augmented With Insights From Personality Traits

The escalating prevalence of mental health disorders, particularly depression, underscores the urgent need for early and accurate detection systems. The widespread use of social media has produced an extensive repository of user-generated content, offering critical insight into individuals’ emotional and psychological states. This development enables novel approaches for automated depression detection. However, existing multimodal depression detection approaches often adopt rigid fusion strategies and disregard individual differences in expressive behavior by adopting generalized, one-size-fits-all frameworks. To bridge this gap, we introduce DeXMAG, a novel personalized depression detection framework that integrates a Cross-Modal Attention mechanism with an Adaptive Gated Fusion strategy. DeXMAG effectively combines textual content, visual imagery, and user-level personality traits to capture the heterogeneous and individualized patterns of depressive expression across modalities. Leveraging domain-specific transformer-based language models and a convolutional neural networks-based model, we extract modality-specific features and fuse them via a novel adaptive gating mechanism. Unlike traditional fusion techniques, our model leverages modality-aware attention signals and gating coefficients to enable flexible and personalized integration of features. An extensive ablation study reveals that the most substantial performance gain occurs when DeXMAG is augmented with insights from Myers–Briggs Type Indicator (MBTI) personality traits in conjunction with textual and visual features. Comprehensive experiments on multimodal social media datasets demonstrate that DeXMAG consistently outperforms unimodal baselines and state-of-the-art approaches, achieving an accuracy of 95.20% and an F1-score of 95.94%. These results reinforce the effectiveness of adaptive gated fusion and personality-aware modeling in advancing robust and personalized depression detection.