A classifier-based test oracle for embedded software

Despite great advances in different software testing areas, one important challenge, achieving an automated test oracle, has been overlooked by academia and industry. Among various approaches for constructing a test oracle, machine learning techniques have been successful in recent years. However, there are some situations in which the existing machine learning based oracles have deficiencies. These situations include testing of applications with low observability, such as embedded software and multimedia software programs. There are also cases in testing embedded software in which explicit historical data in form of input-output relationships is not available, and situations in which the comparison between expected results and actual outputs is impossible or hard. Addressing these deficiencies, this paper proposes a new black box solution to construct automated oracles which can be applied to embedded software and other programs with low observability. To achieve this, we have employed an Artificial Neural Network (ANN) algorithm to build a model which merely requires program's input values as well as corresponding pass/fail outcome, as the training set. We have conducted extensive experiments on several benchmarks. The results manifest the applicability of the proposed approach to software systems with low observability as well as its higher accuracy in comparison to a well-known machine learning based method.