HURON: A Quantitative Framework for Assessing Human Readability in Ontologies

The increasing use of ontologies requires their quality assurance. Ontology quality assurance consists of a set of activities that allow analyzing the ontology, identifying strengths and weaknesses, and proposing improvement actions. Human readability is a quality aspect that improves the use and reuse of ontologies. Human readable content refers to the natural language content consumed by humans and by the growing number of embedding methods applied to ontologies. The ontology community has proposed best practices for human readability, but there is no standardized framework for its evaluation. We aim to provide a framework for analyzing the human readability based on quantitative metrics to support ontology developers’ decisions. We present the HURON framework, which consists of the specification of five quantitative metrics related to the human readability of ontology content and a software tool to implement them. The metrics take into account the number of names, descriptions, or synonyms, and also assess the application of systematic naming conventions and the ‘lexically suggest, logically define’ principle. Target values are provided for each metric to help to interpret them. HURON can also be used to assess compliance with best practices. We have applied HURON to a representative set of biomedical ontologies, the OBO Foundry repository. The results showed that, in general, the OBO Foundry ontologies comply with the expected number of descriptions and names in their classes, and both lexical and semantically formalized contents are aligned. However, most of the ontologies did not follow a systematic naming convention. In general, the ontologies in this repository show adherence to some of the best practices, although areas for improvement were identified. A number of recommendations are made for ontology developers and users.