CSWf: A cloud service workflow system

Accompany with the development of information service and rapid expansion of cloud computing, scientific workflow system is facing challenges of growing size of heterogeneous data, complexity of scientific computing and difficulty of task integration. In this paper, a cloud scientific workflow system-CSWf, which based on Hadoop, NoSQL and Web Service technology, is proposed to make an effective integration of data and service resources in the loosely-coupled cloud service environment. To implement CSWf, a simple but effective cloud service workflow modeling language is designed, and then a reliable workflow engine is developed to parse and schedule the workflow processes, also a distributed execution framework is built to encapsulate workflow jobs and execute workflow for CSWf in the cloud environment. CSWf takes benefits of massive data storage capacity and distributed parallel computing power of cloud computing. It accommodates well on the requirement of modeling, scheduling, coordinating and executing workflows on a distributed workflow system. At the end of this paper, an urban regional air pollution workflow with different size input data on the cluster is run, to measure the performance of CSWf. The result shows that CSWf can significantly improve the efficiency of workflow execution.