Dynamic Network Bandwidth Resizing for Big Data Applications

Big Data concerns processing of large volumes of digital data with high velocity and variety. Big Data technologies allow the analysis of data in real time, which is critical for various eScience applications. In order to meet the growing demand of Big Data applications, the infrastructures must be flexible enough to adapt to the characteristics of the applications. Most of the solutions presented in the literature to support Big Data applications focus on scaling processors and memory to handle a variable demand from applications. In a complementary way, this article targets the problem of adapting the network bandwidth to the amount of data to be transferred to and from the applications in order to improve the performance of the applications. For this purpose, we propose the use link aggregation protocol along with Software-Defined Network capabilities for management of the network flow. Results showed that the proposed approach improves the application's performance by up to 33%.