Selecting and Constraining Metamorphic Relations

Software testing is a critical aspect of ensuring the reliability and quality of software systems. However, it often poses challenges, particularly in determining the expected output of a System Under Test (SUT) for a given set of inputs, a problem commonly referred to as the test oracle problem. Metamorphic Testing (MT) offers a promising solution to the test oracle problem by examining the relations between input-output pairs in consecutive executions of the SUT. These relations, referred to as Metamorphic Relations (MRs), define the expected changes in the output when specific changes are made to the input. Our research is focused on developing methods and tools to assist testers in the selection of MRs, the definition of constraints, and providing explanations for MR outcomes. The research is divided in three parts. The first part focuses on MR collection and description, entailing the creation of a comprehensive repository of MRs from various sources. A standardised MR representation is devised to promote machine-readability and wide-ranging applicability. The second part introduces MetraTrimmer, a test-data-driven approach for systematically selecting and constraining MRs. This approach acknowledges that MRs may not be universally applicable to all test data space. The final part, evaluation and validation, encompasses empirical studies aimed at assessing the effectiveness of the developed methods and validating their suitability for real-world regression testing scenarios. Through this research, we aim to advance the automation of MR generation, enhance the understanding of MR violations, and facilitate their effective application in regression testing.