Context-Aware Mental Health Behaviour Detection from Text using Transformer-Based Language Models

Depression and anxiety are among the world’s leading causes of disability; however, diagnosing them in good time has long been a problem because of restricted accessibility, the stigma attached to them, and the fact that they are diagnosed based on subjective assessment. Specifically, this paper proposes a machine learning based framework to automatically detect mental health conditions by analysing linguistic patterns of text. Two publicly available datasets, CLPsych 2015 (posted to Reddit) and DAIC-WOZ (clinical interviews) were used to compare the results of four models, Logistic Regression with TF-IDF, Support Vector Machine (SVM), Bidirectional Long Short Term Memory (BiLSTM) with GloVe embeddings, as well as Bidirectional Encoder Representations from Transformers (BERT). A 5-fold cross-validation was performed to evaluate the accuracy, precision, recall, $\mathbf{F 1}$-score performance, and their complementary AUC-ROC. Finally, the best achieved results were the BERT model, which had accuracy of $92.4 \%$, precision of $91.8 \%$, recall of $90.7 \%$, F1-score of $91.2 \%$, and AUC-ROC of 0.95, significantly better than BiLSTM (accuracy 87.9%), SVM (accuracy 83.5%) and Logistic Regression (accuracy 81.3%). The training diagnostics show strong convergence per training loss, which reduced to 0.22, and training accuracy to $91 \%$ at five epochs. In addition, the class clusters were well segregated as per the t-SNE visualisations, and the confusion matrix has a low false negative rate ($7.4 \%$), i.e., an outstanding product rate. This result proves that transformer models can detect mental health signals on language data, a domain that can be scaled and interpreted for early screening. This work opens the door for new AI mental health tools that provide clinical support and can intervene proactively in clinical and digital health contexts.