Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Heterogeneous Sources

Conventional RAG approaches, though effective for general tasks, often fail to retrieve precise information for complex questions involving multiple entities, which is called Multi-Entity question answering (QA), such as “How are all IEEE Fellows distributed across distinct research domains by 2024?”. To bridge this gap, we propose an innovative structured retrieval-augmented generation framework that includes semantic KG retrieval driven by large language model (LLM) and a table generation module, which transforms unstructured data into relational table structures. This structured format enables efficient querying via advanced table-based methods, ensuring high precision for intricate, entity-dense questions. Complementing our approach, we develop a tailored benchmark to rigorously assess performance of RAG systems for Multi-Entity QA. Experimental evaluations reveal that our system outperforms state-of-the-art baselines by 11.6% in accuracy, demonstrating the critical role of structured database technology in improving RAG systems for knowledge-intensive applications. This work paves the way for integrating database principles with generative AI to tackle real-world QA challenges requiring precise, multifaceted reasoning.