Detecting Factual Hallucinations in LLMs: A Cross-Domain Dynamic Validation Framework for Fact-Consistency

Although generative pre-training models improve the quality of text generation for large language models, factual error detection becomes more difficult due to three fundamental problems. First, deploying large models increases the risk of errors during task processing. Second, the granularity of factual statements in long texts generated by existing large models is unclear, and the definition of individual facts lacks nuance. Third, the fact-checking process lacks traceable sources of evidence. A large amount of computational resources are utilized to solve these problems, but these resources are insufficient for efficiently solving complex, multi-hop problems. The proposed solution is an adaptive fact-detection framework called FactChecker. It dynamically selects the most appropriate strategy, ranging from the simplest to the most complex large language model (LLM), based on the complexity of the problem. The framework uses small LLMs as classifiers to predict the complexity of the verification process. Thus, FactChecker can identify factual inaccuracies in texts generated by large language models (e.g., ChatGPT). Experiments on three different tasks demonstrate the effectiveness of this approach.