Cross-layer management of a containerized NoSQL data store

Internet-scale services increasingly rely on NoSQL data store technologies for scalable, highly available data persistence. To increase resource efficiency and deployment speed, such services are adopting new deployment models based on container technologies. Emerging container management systems (CMS) offer new models of interaction with containerized applications, exposing goal-oriented management capabilities. In this paper, we study the interaction of a containerized NoSQL data store, RethinkDB, with a popular open-source CMS, Kubernetes, demonstrating that events exposed by the CMS can be leveraged to drive adaptation actions on the NoSQL data store, improving its availability and quality of service. Typical adaptation actions in data stores often involve movement of data (e.g., migrating data replicas to a new node) and thus have high overhead. In this paper we demonstrate that lower-cost adaptation actions based on targeted reconfigurations of replica groups are possible and often offer rapid response to performance degradation (such as when a node experiences a temporary resource shortage). Such reconfigurations can be exercised by a cross-layer management system that bridges between CMS and NoSQL data store. We evaluate a prototype implementation of this management system in the context of Kubernetes and RethinkDB using the Yahoo Cloud Serving Benchmark on Google Container Engine. Our results demonstrate that infrastructure-level events provided by the CMS can drive proactive, low-cost data-store adaptation actions that improve overall system manageability, availability, and performance.