Storage Service Orchestration with Container Elasticity

The advent, popularity and widespread adoption of Cloud-Native architectures and platforms have revolutionized the way in which applications are being built and deployed in the Cloud. Applications designed for Cloud-Native architectures are being migrated from traditional monolithic approach to a Microservices architecture, in which an application is broken down into a collection of small, independent and collaborative services. These applications are dynamically scheduled and managed by Cloud Native platforms. Container Orchestrators (CO), such as Docker, Kubernetes, Cloud Foundry, enable the deployment and management of stateless and scalable microservices. However, as more and more stateful applications and workloads are being deployed in Cloud-Native platforms, these platforms must support the workloads by providing persistent storage to the microservices so that data can be protected and accessed beyond the lifecycle of an ephemeral container. In this paper, we present the design and development of a Cloud-Native storage service orchestration platform by leveraging capabilities of the IBM Ubiquity framework. Our ultimate goal is to provide persistent storage across different container orchestrators and supporting multi-protocol access for Volume management within the storage systems, such as SSH, REST, GPFS native protocol using CLI, NFS and Object. We demonstrate the effectiveness of the storage service orchestration development vision by providing a prototype implementation using Docker and General Parallel Filesystem (GPFS), aiming to provide shared storage volumes to containers orchestrated by Docker. As the volume and variety of microservices continue to grow and to dominate the development and deployments of Cloud applications, we conjecture that the proposed storage service orchestration framework will be a critical enabling building block for supporting elastic, reliable and effective microservice architectures in Cloud datacenters.