Multivariate Motif Detection in Local Weather Big Data

In recent years, there are very frequent reports of disasters attributed to the climate change and there are several reports that these extreme phenomena will further affect people not only as weather disasters but also indirectly with the shortage of natural resources such as water or food due to the climate change. Towards this direction, there is an on-going research that studies weather phenomena by collecting data not only in the surface of the globe but also at the different levels of the atmosphere. Having such a large volume of data, traditional numerical weather prediction models may not be able to assimilate those data and extract knowledge useful for the prediction of extreme phenomena. Thus, analysis of weather data has been transformed into a big data analytics problem which may enable weather scientists to better understand the interrelations of the weather variables and use the knowledge discovered to improve their prediction models. In this context, the current paper proposes a big data analytics methodology that is able to detect all common patterns between different weather variables in neighboring or distant points in a specific time window revealing useful associations between weather variables which is not possible to detect otherwise with the traditional numerical methods. The proposed methodology is based on a data structure that is able to store the magnitude of the weather data in different dimensions and a pattern detection algorithm which is able to detect all common patterns. The experimental results using weather data from the National Oceanic and Atmospheric Administration (NOAA) revealed interesting otherwise unknown patterns in two weather variables for two specific locations that were studied.