Data-Intensive HPC Tasks Scheduling with SDN to Enable HPC-as-a-Service

Advances in Cloud Computing attracted scientists to deploy their HPC applications to the cloud to benefit from the flexibility of the platform such as scalability and on-demand services. Nevertheless, HPC programs can face serious challenges in the cloud that could undermine the gained benefits. This paper first compares the performance of several HPC benchmarks on a commodity cluster and Amazon public cloud to illustrate the confronted challenges. To mitigate the problem, we have introduced a novel approach called ASETS, "A SDN Empowered Task Scheduling System", to schedule data-intensive High Performance Computing (HPC) tasks in a Cloud environment. In this paper, we focus on the implementation and performance analysis of ASETS and its first algorithm called SETSA, (SDN Empowered Task Scheduling Algorithm). ASETS uses the "bandwidth awareness" capability of SDN to better utilize the bandwidths when assigning tasks to virtual machines. This approach aims to improve the performance of HPC programs and provides an efficient HPC-as-a-Service (HPCaaS) platform. The paper briefly describes the architecture and the algorithm, and then focuses on the details of the implementation and performance analysis of ASETS and SETSA. Preliminary results indicate that ASETS delivers substantial performance improvement for HPCaaS as the degree of multi-tenancy increases. This result is significant since it indicates both the users and the cloud service providers can benefit from ASETS.