Quality and Success in Open Source Software: A Systematic Mapping

As the number of available Open Source Software (OSS) and the interest they attract are increasing, numerous product attributes are provided to developers and users for evaluating the quality and success of an OSS. Accordingly, various articles in the literature assess the quality and success of OSS, by using different quality attributes and metrics and different approaches. Though this variety can be considered as a positive indicator of research interest and maturation on one side, it creates a kind of jungle in defining and understanding the terms 'quality' and 'success' on the other side. Based on this challenge, in this study, we targeted a systematic mapping (SM) of the articles on quality and success of OSS. More than 474 articles have appeared in this area between the years 2002 and 2017, and the final pool of 128 articles is obtained by defining and applying inclusion and exclusion criteria. SM was employed to develop a classification scheme and categorized the existing body of articles with respect to five research questions (RQs) on: contribution and research types, quality criteria and metrics, success criteria and metrics, the relation of quality and success, and demographics. We observed that the majority of the articles assess the concept of quality as 'code quality', whereas the concept of success is mostly perceived as 'market success' and/or 'developer activity'. Moreover, the metrics of 'contributing developers/users', and the quality attribute of 'functionality' are the quality criteria most employed in the assessment of success.