DP-BERT: a pre-trained deep language model for depression prediction using microarray data

In recent years, the increasing number of individuals diagnosed with depression and the growing awareness of its impact on modern society have highlighted the significance of accurate depression diagnosis. Microarray data has played a crucial role in uncovering the genetic mechanisms underlying depression. However, existing methods for depression prediction using microarray data often rely on the selection of differentially expressed genes. This approach disregards important information from other genes and is susceptible to batch effects, thereby limiting generalizability and model stability. To address these limitations, we propose DP-BERT, a depression prediction model based on Bidirectional Encoder Representations from Transformers (BERT). DP-BERT follows a pre-training and fine-tuning paradigm, leveraging a large amount of unlabeled microarray data from diverse sequencing platforms for pretraining to extract comprehensive genetic-level representations of psychiatric disorders. Subsequently, supervised fine-tuning is performed for depression prediction. Experimental results demonstrate that the pre-trained model achieves superior performance in depression prediction. The source code can be obtained from https://github.com/CSUBioGroup/DP-BERT.