Data Mining and Analysis of Large Scale Time Series Network Data

Large amounts of data are readily available and collected daily by global networks worldwide. However, much of the real-time utility of this data is not realized, as data analysis tools for very large datasets, particularly time series data are cumbersome. This research presents a comparative study of three data mining tools using a large scale time series dataset from NOAA for analysis and mining. Meteorological data, gathered daily, if used at all, is useful for a very short period of time, both to help determine current weather conditions and to predict upcoming weather events. Current weather prediction methods can only guess at what the conditions will be in the near-term future, approximately one week at a time. The goal of this research project was to take large amounts of archival NOAA weather data and use appropriate data mining algorithms to identify patterns that could help predict future weather events. The results of this work identify the merits of the Rapid Miner tool over Weka and Orange, and provide future direction for data mining on massive data sets gathered from global networks.