Evaluation of virtual machine scalability on distributed multi/many-core processors for big data analytics

Cloud computing makes data analytics an attractive preposition for small and medium organisations that need to process large datasets and perform fast queries. The remarkable aspect of cloud system is that a nonexpert user can provision resources as virtual machines (VMs) of any size on the cloud within minutes to meet his/her data-processing needs. In this paper, we demonstrate the applicability of running large-scale distributed data analysis in virtualised environment. In achieving this, a series of experiments are conducted to measure and analyze performance of the virtual machine scalability on multi/many-core processors using realistic financial workloads. Our experimental results demonstrate it is crucial to minimise the number of VMs deployed for each application due to high overhead of running parallel tasks on VMs on multicore machines. We also found out that our applications perform significantly better when equipped with sufficient memory and reasonable number of cores.