CloudEx: A Novel Cloud-Based Task Execution Framework

In recent years cloud computing has seen steady adoption due to its unique features such as computing resource elasticity, fault-tolerance and utility billing. Cloud computing Infrastructure-as-a-Service (IaaS) enables unique architectures that can dynamically scale and configure computing resources from a catalogue of available features. In addition to provisioning long running homogeneous clusters of Virtual Machines (VMs), it can also be feasible to provision ephemeral and heterogeneous per-job VMs. This is made possible due to the reduced VM startup time and per- minute billing for cloud VMs. In this paper we design and implement CloudEx, a generic and novel framework for executing jobs on public clouds by leveraging the Google Cloud Platform. CloudEx enables users to split jobs into a sequence of smaller tasks that can be distributed using Bin Packing or user-defined algorithm. Additionally, users can specify the VM specification per job or per task, CloudEx then provisions the required VMs, coordinates the job execution and terminates these VMs once the job is completed.