QoS Aware Resource Management for Apache Cassandra

Apache Cassandra is a distributed database of choice when it comes to big data management with zero downtime, linear scalability, and seamless multiple data center deployment. However, resource allocation for the system during deployment is a major issue which could either lead to bad performance or under-utilization of resources. In this paper we describe a QoS-aware architecture that manages resources for the distributed storage system to proactively and dynamically allocate resources for the distributed storage system to ensure that effective resource utilization and deliver performance according to the specified QoS. The architecture uses machine learning techniques to proactively predict and judge the performance of the system and make decisions for effective resource management for the database. In addition, we propose an adaptive sampling mechanism for classification to ensure that the architecture does not impose an unreasonable overhead on the system. We evaluate and provide results using Yahoo! Cloud Serving Benchmark (YCSB) on the modified Cassandra cluster. Our evaluation shows an increase of the overall utilization of the allocated resources without compromising on the required QoS latency for the overall cluster.