Experiences of On-Demand Execution for Large Scale Parameter Sweep Applications on OSG by Swift

Large scale parameter sweep application (PSA) is one of the main grid applications, which may have different characteristics and demands. In this paper, we describe how to use swift to enable the on-demand execution of large scale PSA on open science grid (OSG). The basic on-demand concept means providing appropriate grid resources for the application, which is decided by the characteristics and demands of the application. So we can get high reliability, efficiency, and scalability for large scale independent PSA jobs on OSG. The main on-demand policies include: trust based site selection and pre-selection; scheduling policy on-demand configuration; clustering for small jobs; adaptive execution and automatic data staging; divide and conquer for the scalability. Some usage examples of swift for executing large scale PSA are presented, such as dock, blast. The experimental results for the performance of different policies are presented, with a benchmarking workload size of 10,000 jobs.