High Fluctuation Based Recursive Segmentation for Big Data

In recent years, the world has seen a huge increase in the amount of information being shared via the internet, cloud computing and mobile devices. This huge amount of data being collected and captured has triggered the need to be analyzed thoroughly, to predict future behaviors and evaluate their impact on decision-making in various industry fields. Although there is currently a big hype around using this enormous amount of data often called big data for better future business strategies, there is still yet a need for a quantitative definition of data bigness and the importance to make use of both past and latest data behavior to tackle the considerable challenge the speed at which these data are generated in addition to the variety of data collected on medium/long term forecasting. This paper aims to first provide a quantitative definition of data bigness, then propose a dynamic window segmentation algorithm to help mitigate the challenges of fast pace big data velocity on the implementation of big data analysis within businesses for medium to long term forecasting. Thus, in this research, upon review of existing literatures, we will introduce a dynamic segmentation approach that makes use of dependency algorithms such as recursive segmentation in addition to high fluctuation windows to better optimize big data prediction models subject to limitation constraint such as the maximum volume of data to be processed. The paper provides a detailed implementation framework and evaluates its effectiveness in regard to existing prediction methods.