GDIA: A Scalable Grid Infrastructure for Data Intensive Applications

The applications in many scientific fields, like bioinformatics and high-energy physics etc, increasingly demand the computing infrastructures can provide more computing power and support huger amount of distributed data. GDIA, built on top of CSF4 meta-scheduler and Gfarm data grid, is a scalable grid infrastructure for data intensive applications. In this paper, we presented the architecture of GDIA and the new enhancements to CSF4 in GDIA. First, a flexible user proxy delegation mechanism was introduced to enable a job running with a full proxy. With the enhancement, the jobs can access the grid services with strict security requirement like Gfarm. Secondly, we redesigned CSF4's resource manager service to support alternative protocols other than WS GRAM. At last, we discussed the scheduling issues in grids. GDIA is able to coordinate heterogeneous clusters belonging to different VOs via centralized or decentralized model. In current, GDIA has been deployed on PRAGMA's grid test bed successfully to schedule data intensive applications.