Enhancing RAG Performance by Representing Hierarchical Nodes in Headers for Tabular Data

Recent developments in retrieval-augmented generation (RAG) techniques have aimed at integrating structured tabular data with external data sources. Nevertheless, because existing approaches have difficulty effectively retrieving and reasoning from diverse and complex table structures, there is a growing demand for innovative methods that surpass traditional retrieval paradigms. This study introduced hierarchical node-based header representation models designed to enhance the processing and integration of external table data, addressing the key limitations of existing RAG techniques when working with structured table sources. By classifying table types designed for the structural complexity and the level of domain knowledge required for their interpretation, this study explores the impact of using embedded information to improve retrieval accuracy. The findings of this study demonstrate that presenting richer and more meaningful information within table headers significantly enhances RAG performance. This improvement is attributed to the utilization of table header representations, which allow for a more precise identification of hierarchical node-based headers during the table retrieval process. Furthermore, this study confirms that vector database achieves greater accuracy when employing a distributed table collection (DTC) structure than a unified context collection (UCC). These results indicate that the structure of the information extracted from text and tables varies considerably. Our experiment results highlight that separating and organizing such information using the DTC structure proved to be an effective strategy for improving the accuracy of table-specific responses.