Resilient Edge Data Management Framework

Transferring and processing huge amounts of data in the cloud can violate the low latency requirements of modern IoT applications, considering underlying network infrastructure limitations. Edge data analytics is a promising solution. However, edge resources have usually less computational capabilities than cloud nodes, resulting in a higher failure rate of IoT systems. Consequently, near-real-time decisions are often based on limited and incomplete data. State-of-the-art solutions, such as operational/workload flows, data reduction, reconstruction, focus mostly on resource and network optimization, while approaches for incomplete data recovery employ a single specific method, despite diverse data characteristics. Data quality impact on accuracy of the decision-making processes is often neglected. We propose EDMFrame, a framework featuring a generic mechanism for recovery of multiple gaps in incomplete datasets, using single-technique recovery (STR) and multiple-technique recovery (MTR) involving projection recovery maps (PRMs). We further devise an adaptive storage management mechanism for reducing data stored at the edge, keeping only the data necessary for predictive analytics. We conduct experiments using time series from smart buildings, (i) automatically recovering various multiple gaps and reducing errors up to 65.48 percent with MTR compared to STR; (ii) reducing amounts of data stored to 39.9 percent on average, keeping prediction accuracy around 98.83 percent.