Pythia: Facilitating Access to Internet Data Using LLMs and IYP

Internet data analysis is essential for policymakers and regulators to make informed decisions. Despite the various datasets made available by the research community, the analysis of these datasets is challenging due to their various formats and required analysis tools. The Internet Yellow Pages (IYP) database is designed to simplify Internet data analysis by combining many datasets into a single format. Hence, IYP enables users to retrieve data from various Internet datasets using a single querying language called Cypher. However, learning Cypher and understanding IYP’s schema is still challenging. Large Language Models (LLMs) offer the potential to generate Cypher queries from English text. In this paper, we assess the ability of different LLMs to generate Cypher queries. We introduce CypherEval, a dataset for evaluating LLM-generated Cypher queries, and we propose a methodology to benchmark LLMs for IYP. Finally, we present Pythia, a system for the generation of IYP Cypher queries.