Large Model Fine-tuning for Suicide Risk Detection Using Iterative Dual-LLM Few-Shot Learning with Adaptive Prompt Refinement for Dataset Expansion

An approach to detecting suicide risk in social media posts is presented, addressing the challenges of limited and imbalanced datasets. The proposed workflow combines large language models (LLMs), few-shot learning, and expert-supervised prompt optimization. To address class imbalance, a novel Iterative Dual-LLM Few-Shot Learning with Adaptive Prompt Refinement (IDFL-APR) method for dataset expansion is introduced. This method employs two LLMs: one for classifying posts using few-shot learning, and another for dynamically optimizing prompts, with expert oversight to prevent overfitting. The optimized LLM then identifies high-confidence samples from unclassified data, focusing on underrepresented categories. Back-translation techniques are further applied to enhance textual diversity and achieve dataset balance across all categories. Subsequently, the bloomz-3b model is fine-tuned using optimized hyperparameters, implementing an active learning strategy to iteratively augment the training set. The proposed approach significantly enhances suicide risk detection accuracy. The best-performing model achieved a weighted F1-score of 0.7154 on the test set provided by the Suicide Ideation Detection on Social Media Challenge, a track of the IEEE Big Data 2024 BigData Cup. These results demonstrate a robust solution to the inherent challenges in suicide detection tasks.