Evaluation metrics for automatically constructed concept maps

Concept maps are knowledge visualization tools that allow representing the text or domain at a conceptual level. They reflect the systemic relations between the key concepts of the text and thereby contribute to a deeper understanding of its ideas, save time spent on reading and analysis. However, the very process of creating concept maps is laborious and time-consuming. At the same time, with the rapid growth of digital reading services, the automatic construction of concept maps attracts an increasing intensive research. Against that background, comparison and evaluation of methods for automatic construction of concept maps are of great importance. In this paper, we discuss popular evaluation metrics for automatically created concept maps and propose our new metric based on network centrality analysis. We test all the considered metrics by comparing an automatic concept map with a reference concept map developed manually by experts. Experiments show that our proposed metric complements existing metrics by providing information about significance degrees of concepts and relations.