Evaluating Reliability-Testing Usage Models

Testing the reliability of an application usually requires a good usage model that accurately captures the likely sequences of inputs that the application will receive from the environment. Markov usage models and their variations have been found to be well suited for generating test cases that are statistically close to what the application is expected to receive when in production. In this article, we study the specific case of web applications. We present an evaluation method for estimating the accuracy of various reliability-testing usage models. The method is based on comparison between observed users' traces and traces inferred from the usage model. Our method gauges the accuracy of the reliability-testing usage model by calculating the sum of goodness-of-fit values of each traces and scaling the result between 0 and 1.