Towards Greening MapReduce Clusters Considering Both Computation Energy and Cooling Energy

Increased processing power of MapReduce clusters generally enhances performance and availability at the cost of substantial energy consumption that often incurs higher operational costs (e.g., electricity bills) and negative environmental impacts (e.g., carbon dioxide emissions). There exist a few greening methods for computing clusters in the literature that focus mainly on computational energy consumption leaving cooling energy, which occupies a significant portion of the total energy consumed by the clusters. To this extent, in this article, we propose a machine learning-based approach that reduces the total energy consumption of a MapReduce cluster considering both computational energy and cooling energy. Our approach predicts the number of machines that results in minimum total energy consumption. We perform the prediction through applying different machine learning techniques over year-long data collected from a real setup. We evaluate performance of our approach through both real test-bed experimentation and simulation. Our evaluation reveals that our approach achieves substantial reduction in total energy consumption compared to other state-of-the-art alternatives while experiencing marginal performance degradation in a few cases.