Jetstream2: Research Clouds as a Convergence Accelerator

Over the past decade, the convergence of cloud and high-performance computing (HPC) has undergone significant movement. We explore the evolution, motivations, and practicalities of establishing on-premise research cloud infrastructure and the complementary nature with HPC and commercial resources; under the belief that research clouds serve a unique role within research and education (R&E) as a convergence accelerator. This role is highlighted through exploring the design tradeoffs in architecting research clouds versus HPC resources, focusing on the balance among utility, availability, and hardware utilization. The discussion provides insights from experiences with the U.S. National Science Foundation-supported Jetstream and Jetstream2 systems, showcasing convergence technologies and challenges. A variety of real-world use cases that show the interplay between these computing paradigms are provided; we explore their use in R&E for interactive and iterative development, as an on-ramp to large-scale resources, as a powerful tool for education and workforce development, and for domain-specific science gateways.