On-Demand Capacity Provisioning in Storage Clusters Through Workload Pattern Modeling

Internet of Things (IoT), which is the inter-networking of a wide variety of physical devices, is widely used in our daily life. The exponential increase in the number of diverse devices has resulted in a significant increase in the volume, variety, velocity, and veracity of data (i.e., big data). These data present a large requirement on modern storage systems both for capacity and scale, and energy cost has become a critical problem. For storage clusters, much research effort has been invested in alleviating this problem by providing suitable resource capacity (i.e., on-demand providing). However, it is challenging to match the offered resource capacity with the real system workloads, thus resulting in a violation of service level agreement. By considering a storage cluster as a queueing system, this paper proposes a QoS-oriented capacity provisioning mechanism. Based on workload features, the mechanism models the pattern of current workloads as a suitable queueing model. In accordance with the model, our mechanism can well forecast the actual resource capacity demand without violating the service level agreement, and then offer the required resource capacity in terms of the real workloads. Experimental results demonstrate that the proposed mechanism significantly reduces the energy consumption of a typical storage cluster, while meeting the QoS requirements. It also significantly outperforms two classic and two state-of-the-art capacity provisioning mechanisms.