Leveraging and Adapting ExoGENI Infrastructure for Data-Driven Domain Science Workflows

In this paper, we present our ongoing work on a novel use of networked cloud infrastructures like GENI for running adaptive domain science applications. We specifically report our recent experience at the SC'13 conference with showcasing a dynamically adaptable cloud infrastructure driven by the demand of a data-driven scientific workflow. Our work used resources from ExoGENI - a Networked Infrastructure-as-a-Service (NIaaS) testbed funded through NSF's Global Environment for Network Innovation (GENI) project. We used on-ramps to compute and data resources in the RENCI SC'13 booth to a large dynamically provisioned 'slice' spanning multiple ExoGENI cloud sites that were interconnected using dynamically provisioned connections from Internet2, NLR and ESnet. The slice was used to execute a scientific workflow driven from a computer in the RENCI SC'13 booth connected to the slice via SCinet. A closed-loop control mechanism leveraging a monitoring infrastructure based on persistent queries adapted the slice to the demands of the workflow as it executed.