A Joint LLM-KG System for Disease Q&A

Medical question answer (QA) assistants respond to lay users’ health-related queries by synthesizing information from multiple sources using natural language processing and related techniques. They can serve as vital tools to alleviate issues of misinformation, information overload, and complexity of medical language, thus addressing lay users’ information needs while reducing the burden on healthcare professionals. QA systems, the engines of such assistants, have often used large language models (LLMs) or knowledge graphs (KG), though the approaches could be complementary. LLM-based QA systems excel at understanding complex questions and providing well-formed answers but are prone to factual mistakes. KG-based QA systems, which represent facts well, are mostly limited to answering short-answer questions with pre-created templates. While a few studies have used both LLM and KG for text-based QA, the approaches are still prone to incomplete or inaccurate answers. Extant QA systems also have limitations in terms of automation and performance. We address these challenges by designing a novel, automated disease QA system named Disease Guru–Long-Form Question Answer (DG-LFQA), which effectively utilizes both LLM and KG techniques through a joint reasoning approach to answer disease-related questions appropriate for lay users. Our evaluation of the system using a range of quality metrics demonstrates its efficacy over related baseline systems.