Simulating Application Workflows and Services Deployed on the European Grid Infrastructure

This paper presents an end-to-end SimGrid-based simulation of a Monte-Carlo computation deployed on the European Grid Infrastructure. We describe a simulation framework allowing to replay executions from real traces. Middleware services, namely the file catalog, storage elements, the DIRAC pilot-job system, and the MOTEUR workflow engine are simulated by SimGrid processes. The deployment of pilot jobs, performed on EGI by the gLite WMS and batch queues, is simulated by a random selection of platform hosts, and a matching of their latencies and failure times with real traces. The Monte-Carlo application workflow is calibrated to address performance discrepancies between the real and simulated network and CPUs. The simulation is evaluated against real executions of the GATE Monte-Carlo application. Results show that SimGrid can be used to study the performance of applications running on EGI. Simulated and real make spans are consistent, and conclusions drawn in production about the influence of application parameters such as the check pointing frequency and the number of mergers are also made in simulation. This opens the door to better and faster experimentation on production grids.