Textual-dominated Multimodal Depression Detection

Depression poses a major threat to an individual's overall physical and psychological health, as well as their social functioning. Consequently, it is urgent to design an effective automated diagnostic technique for depression. Recent studies have revealed that multimodal approaches produce better detection performance than unimodal methods. To this end, this paper introduces a multimodal depression detection framework using visual and textual data. Initially, separate networks are employed to extract their shallow features, aiming to capture the diversity across various modalities. The visual features are then integrated into the textual features through a newly designed Textual-dominated Cross-modal Fusion module to highlight the information transmitted by the textual features. Finally, the Multilayer Perceptron-based Hybrid Unit is designed to facilitate deep feature interaction, improving the overall detection performance. Our method was assessed on the NCUDID dataset and achieved promising experimental results with a value of 3.41 for the MAE and 4.59 for the RMSE.