Transfer Learning using IndoBERT with Long Short-Term Memory Classifier for Detection of Suicide Ideation Themed Indonesian Twitter Posts

Indonesia’s lack of profiling on suicide and an overall negative mindset towards mental health issues contributes to a lack of effective suicide prevention methods, particularly in the scope of social media as a platform to express suicide ideations, with little to no studies having been made regarding building robust models aimed at suicide ideation detection of Indonesian language posts. This paper presents eight distinct text classification models fine-tuned to effectively detect suicide ideation in Indonesian language Twitter posts by combining four variants of pre-trained IndoBERT models with a convolutional neural network (CNN) or long short-term memory (LSTM) network through transfer learning. Each of these eight models are trained using a dataset of 17464 Twitter posts collected through querying Twitter using Indonesian phrases comprising suicide and non-suicide themed words, which is then annotated and preprocessed through lemmatization, emoji, user tag, and link removal. It is found that an IndoBERT large-p1 pre-trained model combined with a LSTM network using transfer learning offers the highest performance in the detection of suicide ideation in Indonesian language Twitter posts, boasting a $91.41 \%$ accuracy rate, 0.9570 recall score, 0.8830 precision score, 0.9185 F-score, and 0.9675 receiver operating characteristic (ROC) area under curve (AUC) score, achieving high overall performance and minimizing undetected suicidal posts. This is attributed to IndoBERT large’s high number of parameters, coupled with LSTM’s capability of storing long-spanning features within the dataset, allowing the model to effectively understand context within a post and differentiate suicidal and non-suicidal Twitter posts.