Energy Efficient Big Data Networks: Impact of Volume and Variety

In this paper, we study the impact of big data's volume and variety dimensions on energy efficient big data networks (EEBDN) by developing a mixed integer linear programming (MILP) model to encapsulate the distinctive features of these two dimensions. First, a progressive energy efficient edge, intermediate, and central processing technique is proposed to process big data's raw traffic by building processing nodes (PNs) in the network along the way from the sources to datacenters. Second, we validate the MILP operation by developing a heuristic that mimics, in real time, the behavior of the MILP for the volume dimension. Third, we test the energy efficiency limits of our green approach under several conditions where PNs are less energy efficient in terms of processing and communication compared to data centers. Fourth, we test the performance limits in our energy efficient approach by studying a “software matching” problem where different software packages are required to process big data. The results are then compared to the classical big data networks (CBDN) approach where big data is only processed inside centralized data centers. Our results revealed that up to 52% and 47% power saving can be achieved by the EEBDN approach compared to the CBDN approach, under the impact of volume and variety scenarios, respectively. Moreover, our results identify the limits of the progressive processing approach and in particular the conditions under which the CBDN centralized approach is more appropriate given certain PNs energy efficiency and software availability levels.