Locality Aware Traffic Distribution in Apache Storm for Energy Analytics Platform

Recently, as energy facilities generate large volume of data quickly, technologies which process such big data in real time become attractive research fields. In this paper, we propose a novel grouping method that efficiently distributes heavy data traffic by considering locality in terms of data transmission between nodes in Apache Storm. Although recent Storm provides a few grouping methods, none of them can properly support locality. In this paper, we first derive problems of existing grouping methods, then define requirements to solve those problems, and finally propose a locality-aware grouping to meet those requirements. We also empirically compare the proposed locality-aware grouping with the existing shuffle grouping and local-or-shuffle grouping to verify the superiority of our method. Experimental results show that our grouping method can be used a large scale data analysis for the efficient distribution of heavy volume of energy data traffic.