By Topic

Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Anda, B.C.D. ; Dept. of Inf., Univ. of Oslo, Oslo ; Sjoberg, D.I.K. ; Mockus, A.

The scientific study of a phenomenon requires it to be reproducible. Mature engineering industries are recognized by projects and products that are, to some extent, reproducible. Yet, reproducibility in software engineering (SE) has not been investigated thoroughly, despite the fact that lack of reproducibility has both practical and scientific consequences. We report a longitudinal multiple-case study of variations and reproducibility in software development, from bidding to deployment, on the basis of the same requirement specification. In a call for tender to 81 companies, 35 responded. Four of them developed the system independently. The firm price, planned schedule, and planned development process, had, respectively, ldquolow,rdquo ldquolow,rdquo and ldquomediumrdquo reproducibilities. The contractor's costs, actual lead time, and schedule overrun of the projects had, respectively, ldquomedium,rdquo ldquohigh,rdquo and ldquolowrdquo reproducibilities. The quality dimensions of the delivered products, reliability, usability, and maintainability had, respectively, ldquolow,rdquo "high,rdquo and ldquolowrdquo reproducibilities. Moreover, variability for predictable reasons is also included in the notion of reproducibility. We found that the observed outcome of the four development projects matched our expectations, which were formulated partially on the basis of SE folklore. Nevertheless, achieving more reproducibility in SE remains a great challenge for SE research, education, and industry.

Published in:

Software Engineering, IEEE Transactions on  (Volume:35 ,  Issue: 3 )