Analyzing the reliability of simulated distributed systems using Metamorphic Testing

Simulation is widely adopted by the research community to analyze and study complex systems. It is based on the idea of creating a model representing the target system under study, so that the experiments can be executed over the model instead of the target system. However, since the model is a simplification of a real-world system, the obtained results entail an accuracy loss, which makes determining the reliability of the experiments a complex task. Testing can be applied to check the correctness of systems. Thus, an oracle is used to determine if a test is correct or not. In the field of simulation, an oracle can be applied to determine the reliability of the results, but in most cases, the oracle is not available or is computationally too expensive to be applied. In this work, we propose to use metamorphic testing to detect faults in simulated distributed systems. In essence, we use meta-morphic relations - representing the relevant properties of the system under study - as an oracle. Thus, the results provided are contrasted against these relations to determine their reliability. In order to show the applicability of this approach, we have modelled different distributed systems architectures using the SIMCAN sim-ulator and a high performance application that is executed over the models.