Methodology for statistical analysis comparing the algorithms performance: case of study in virtual environments in private cloud computing

The aim of this work is to propose a methodology that seeks to discover how, when and as the increased performance of the algorithms in virtual environments is determined by the environment configuration and how the configuration parameters can influence each other, and finally, discover using statistical methods which settings of virtual environment achieve the best results on average. The experimental design is a pre-established set of tests using scientific and statistical criteria mainly, in order to determine the influence of various factors on the results (metric) of a system or process, identifying and observing the reasons that led to change in the expected value. The planning that was used is factorial planning 34, where each factor (core, memory, operating system and virtual machine) were varied in three levels. Tested operating systems were Ubuntu 14.04 64bit, CentOS 7.0 64bit and Windows 8.0 64bit; and virtual machines were tested KVM, Xen and VMware. Data were collected and analyzed using analysis of variance. The results show that the major analyzed factors changes the algorithm performance, but they cannot be analyzed separately because there are also significant interactions belonging to these factors. At a 5% significance level, analysis of variance showed that the core interactions: memory, memory with OS, memory with VM and OS with VM, all these factors impact the runtime of the analyzed algorithm.