Fine-Tuning the BERT Model to Predict Depression and Anxiety Using Multi-Labeled Twitter Data

Studying mental health through social media data has become an emerging area of research, notably for the detection of depression and anxiety. In this regard, many researches have been conducted, yielding very satisfactory results (e.g., [1]–[3]). However, most of these studies have addressed each of these two mental disorders separately. This is due to the overlap of symptoms associated with depression and anxiety that makes distinguishing between them significantly challenging. Based on this context, this work leverages pretrained large language models (LLMs) to develop efficient multi-class models for predicting both depression and anxiety. For this purpose, a small, multi-labeled Twitter dataset is first constructed. Then, a pre-trained BERT-based uncased model is fine-tuned for six epochs on the dataset, resulting in a model referred to as DAC-BERT. Finally, the DAC-BERT model is evaluated and compared against both hybrid deep learning models and other LLM-based models. The obtained results show that DAC-BERT model outperforms existing approaches, achieving accuracies of up to 97.20% on a multi-labeled dataset containing normal, depressive, and anxious tweets, and up to 96.50% on a dataset with only normal and depressive tweets. These findings highlight the promising potential of fine-tuning LLMs for predicting mental health disorders, particularly depression and anxiety.