A Key Performance Indicator Quality Model and Its Industrial Evaluation

Background: Modern software development companies increasingly rely on quantitative data in their decision-making for product releases, organizational performance assessment and monitoring of product quality. KPIs (Key Performance Indicators) are a critical element in the transformation of raw data (numbers) into decisions (indicators). The goal of the paper is to develop, document and evaluate a quality model for KPIs - addressing the research question of What characterizes a good KPI? In this paper we consider a KPI to be "good" when it is actionable and supports the organization in achieving its strategic goals. We use an action research collaborative project with an infrastructure provider company and an automotive OEM to develop and evaluate the model. We analyze a set of KPIs used at both companies and verify whether the organization's perception of these evaluated KPIs is aligned with the KPI's assessment according to our model. The results show that the model organizes good practices of KPI development and that it is easily used by the stakeholders to improve the quality of the KPIs or reduce the number of the KPIs. Using the KPI quality model provides the possibility to increase the effect of the KPIs in the organization and decreases the risk of wasting resources for collecting KPI data which cannot be used in practice.