Enhanced Heterogeneous Cloud: Transparent Acceleration and Elasticity

This paper presents ORIAN, a fully-managed Platform-as-a-Service (PaaS) for deploying high-level applications onto large-scale heterogeneous cloud infrastructures. We aim to make specialised, accelerator resources in the cloud accessible to software developers by extending the traditional homogeneous PaaS execution model to support automatic runtime management of heterogeneous compute resources such as CPUs and FPGAs. In particular, we focus on two mechanisms: transparent acceleration, which automatically maps jobs to the most suitable resource configuration, and heterogeneous elasticity, which performs automatic vertical (type) and horizontal (quantity) scaling of provisioned resources to guarantee QoS (Quality of Service) objectives while minimising cost. We develop a prototype to validate our approach, targeting a hardware platform with combined computational capacity of 28 FPGAs and 36 CPU cores, and evaluate it using case studies in three application domains: machine learning, bioinformatics, and physics. Our transparent acceleration decisions achieve on average 96% of the maximum manually identified static configuration throughput for large workloads, while removing the burden of determining configuration from the user; an elastic ORIAN resource group provides a 2.3 times cost reduction compared to an over-provisioned group for non-uniform, peaked job sequences while guaranteeing QoS objectives; and our malleable architecture extends to support a new, more suitable resource type, automatically reducing the cost by half while maintaining throughput, and achieving a 23% throughput increase while fulfilling resource constraints.