Domain-Specific Question Answering System of Large Language Model Enhanced by Knowledge Graph

Large Language Models (LLMs) excel in general text tasks but struggle with domain-specific knowledge, leading to knowledge deficiency or forgetting, resulting in hallucination problems. To address this, we propose a framework that integrates knowledge graphs with LLM for collaborative reasoning, taking the medical field as an example. The system comprises four modules: (1) A Question Filter using FastText with iterative training to classify queries; (2) Named Entity Recognition (NER) via LoRA-based fine-tuning of Qwen2.5-3B/7B-Instruct; (3) Text Similarity Retrieval enhanced by Cluster-Simple Contrasive Sentence Embedding (Cluster-SimCSE), which adds clustering loss to SimCSE for Knowledge Graph (KG) entity matching; (4) KG Reasoning Subgraph Generation using node-path and neighborbased strategies to guide LLM responses. Evaluation experiments are conducted on each module and the effectiveness of the whole system is tested on CMCQA and ExplainCPE datasets. The results show improvements over baseline models, verifying the feasibility and robustness of our KG-LLM system which can enhance the accuracy and comprehensiveness of question answering in the LLM field.