The Role of Environment Assertions in Requirements-Based Testing

Software developers dedicate a major portion of their development effort towards testing and quality assurance (QA) activities, especially during and around the implementation phase. Nevertheless, we continue to see an alarmingly increasing trend in the cost and consequences of software failure. In an attempt to mitigate such loss and address software issues at a much earlier stage, researchers have recently emphasized on the successful coordination of requirements engineering and testing. In addition, the notion of requirements-based testing (RBT) has also emerged with a focus on checking the correctness, completeness, unambiguity, and logical consistency of requirements. One seminal work points out that requirements reside in the environment which is comprised of certain problem domain phenomena. Environmental assertions, which connect some of these phenomena in the indicative mood, play a key role in deciding whether a software solution is acceptable. Despite that requirements are located in the environment, little is known about if and how the environment assertions would impact testing and QA activities. In order to address this gap, we present a detailed empirical study, with 114 developers, on the prominence of environment assertions in RBT. Although the results suggest that paying attention to correct, complete, and useful environment assertions has a positive impact on RBT, developers often face difficulty in formulating good assertions from scratch. Our work, to that end, illuminates the potential usefulness of automated support in generating environment assertions.