Bootstrapping Text-to-SQL Resources for Knowledge Graph Question Answering

As Large Language Models continue to grow in size and sophistication, more benchmarks are needed to assess their capabilities. Benchmarks are crucial for measuring progress and pushing the scientific community forward. However, they are often expensive and exploitative to build. In this paper, we demonstrate how to translate the rich and complex existing resources for SQL and relational databases that have been used by the database community, for SPARQL and Knowledge Graph Question Answering (KGQA). We describe how we bootstrapped the largest and most complex KGQA benchmark to date, explore the complexities of converting a relational database into a knowledge graph and describe the methods used to learn rich schema information in the KG from its relational counterpart using a query workload-based analysis. In addition, we show how this method can be used to bootstrap any existing SQL benchmark for KGQA by translating the newly created ScienceBenchmark dataset into a KGQA dataset.