GPT-4: A Stochastic Parrot or Ontological Craftsman? Discovering Implicit Knowledge Structures in Large Language Models

Ontologies are representational artifacts that purport to accurately portray the aspect of reality under the purview of the ontologists laboring upon them. Ontologies exist in a spectrum of formality, from lexical thesauri to knowledge graphs, to collections of statements of first-order logic. The recent proliferation of Large Language Models (LLMs) has brought to bear interactive “knowledge bases” with general awareness of most things. As ontologists create ontologies from their understanding of reality; and as LLMs, presumably, possess some “understanding” of reality, embedded in their vector matrices corresponding to lexical terms from massive quantities of learned texts, a question is posed: what form of ontology can an LLM create when prompted about some novel facet of reality, without explicitly asking it for an ontology? I.e., will an LLM categorize things into bins, or a subsumption hierarchy, or perhaps something else? LLMs, as they are understood, respond when prompted with the most likely response, because they are predictors of next tokens, i.e., they are stochastic parrots. In any case, it is posited that, if prompted without any explicit request for an ontology, an LLM can produce an ontology of novel form, effectively granting insight into the “understanding” an LLM has of the world, as all humans possess an understanding of the world that ontologies are based upon. This paper explores the use of the flagship LLM, GPT-4, in forming an ontology of a novel domain.