Verification and Correction of the Actual Response Content of a Large Language Model by Publicly Available Software Tools

In this paper, we investigated the problem of validity of facts given by a large language model. The main goal of the paper is to develop a module that realizes the fact checking of facts provided by large language models within limited resources. The coreference resolution, evidence retrieval and semantic similarity problems were solved within the scope of the work. External data were used for fact checking Google and Wikipedia. The performance of the modules was tested on data containing short biographies of people of different nationalities and different degrees of fame, the main requirement being the presence of a personal Wikipedia article about the person. The key point in the text factual accuracy evaluation algorithm is the process of splitting the text into short statements containing only one fact. This resulted in two modules with external knowledge retrieval from Google with a metric on test data F1-weighted 0.53 and Wikipedia F1-weighted 0.65.