Measuring software dependability by robustness benchmarking
Mukherjee, A.; Siewiorek, D.P.
Software Engineering, IEEE Transactions on
Volume 23, Issue 6, Jun 1997 Page(s):366 - 378
Digital Object Identifier 10.1109/32.601075
Summary:Inability to identify weaknesses or to quantify advancements in
software system robustness frequently hinders the development of robust
software systems. Efforts have been made to develop benchmarks of
software robustness to address this problem, but they all suffer from
significant shortcomings. The paper presents the various features that
are desirable in a benchmark of system robustness, and evaluates some
existing benchmarks according to these features. A new hierarchically
structured approach to building robustness benchmarks, which overcomes
many deficiencies of past efforts, is also presented. This approach has
been applied to building a hierarchically structured benchmark that
tests part of the Unix file and virtual memory systems. The resultant
benchmark has successfully been used to identify new response class
structures that were not detected in a similar situation by other less
organized techniques
View citation and abstract |