A Novel Prompt Engineering Framework of Large Language Models for Detecting Gout Flares

Gout is a prevalent condition impacting human health globally, frequently causing acute flares that necessitate emergency department (ED) visits. Despite the high incidence of ED encounters for gout, patients often fail to receive adequate continuity of care after discharge. Natural language processing (NLP) models have been employed to identify gout flares during ED visits to provide appropriate outpatient gout care. However, their effectiveness is constrained by the limited availability of annotated data. Traditional models, which rely on large, labeled datasets for training, face significant challenges in adapting to the resource-scarce environment of clinical settings. In this study, we leverage large language models (LLMs) to detect gout flares based on chief complaints, which are often the first available clinical information during ED visits. To enhance LLM performance, we propose a novel prompt engineering framework that systematically improves detection accuracy through the development of tailored prompts. Specifically, we introduce a generic prompt and refine it using gout flare detection guidelines to create a guideline-based prompt. Further, we analyze the errors from the guideline-based prompt and incorporate these insights to develop an error-case-refined prompt. Results from two gout datasets demonstrate that with well-designed prompts, LLMs outperform fine-tuned BERT models in detecting gout flares, even with a limited number of annotated cases. These findings highlight the immense potential of LLMs in gout flare and also other clinical applications, paving the way for improved performance through continued development and domain-specific adaptation.