Towards Validating Complexity-Based Metrics for Software Product Line Architectures

Software product line (PL) is an approach that focuses on software reuse and has been successfully applied for specific domains. The PL architecture (PLA) is one of the most important assets, and it represents commonalities and variabilities of a PL. The analysis of the PLA, supported by metrics, can be used as an important indicator of the PL quality and return on investment (ROI). This paper presents the replication of a controlled experiment for validating complexity metrics for PLAs. In particular, in this replication we are focused on evaluating how subjects less-qualified than the subjects from the original experiment evaluate complexity of a PLA by means of generated specific products. It was applied a PLA variability resolution model of a given PL to a sample of subjects from at least basic knowledge on UML modeling, PL and variability management. Apart of the selection of different subjects, the same original experiment conditions were kept. The proposed PLA complexity metrics were experimentally validated based on their application to a set of 35 derived products from the Arcade Game Maker (AGM) PL. Normality tests were applied to the metrics observed values, thus, pointing out their non-normality. Therefore, the non-parametric Spearman's correlation ranking technique was used to demonstrate the correlation between the CompPLA metric and the complexity rate given by the subjects to each derived product. Such a correlation was strong and positive. The results obtained in this replication shown that even less-qualified subjects, compared to the subjects from the original experiment, are able to rate the complexity of a PLA by means of its generated products, thus corroborating the results of the original experiment and providing more evidence that the composed metric for complexity (CompPLA) can be used as a relevant indicator for measuring the complexity of PLA based on their derived products.