Meta data retrieval for data infrastructure via RAG

Data infrastructure plays a vital role as the cornerstone supporting the operations of modern society. Nevertheless, data characterization methods within this infrastructure encounter challenges related to inefficiency, high costs, and the management of fuzzy and complex retrieval requirements. To tackle these obstacles, this paper introduces a highly efficient approach for characterizing digital objects called DOR-RAF. This methodology integrates Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to enhance the intelligent retrieval experience for users. It presents an interactive retrieval mechanism that accurately identifies users’ true needs through iterative dialogues, effectively handling user queries despite their initial vagueness. The experimental results show that DOR-RAF outperforms the traditional metadata-based retrieval methods in terms of F1 Score and other metrics on the task of retrieving digital objects in a data infrastructure scenario. Meanwhile, the experiments show that the development of data characterisation methods using RAG technology has a broad prospect, and confirm that DOR-RAF outperforms traditional RAG methods in terms of Context Precision, Answer Correctness, etc. in the task of data characterisation.