An Edge-Based Framework for Enabling Data-Driven Pipelines for IoT Systems

Due to the proliferation of the Internet of Things (IoT) paradigm, the number of devices connected to the Internet is growing. These devices are generating unprecedented amounts of data at the edges of the infrastructure. Although the generated data provides great potential, identifying and processing relevant data points hidden in streams of unimportant data, and doing this in near real time, remains a significant challenge. Existing stream processing platforms require the data to be transported to the cloud for processing, resulting in latencies that can prevent timely decision making or may reduce the amount of data processed. To tackle this problem, we designed an IoT Edge Framework, called R-Pulsar, that extends cloud capabilities to local devices and provides a programming model for deciding what, when, and where data get collected and processed. In this paper, we discuss motivating use cases and the architectural design of R-Pulsar. We have deployed and tested R-Pulsar on embedded devices (Raspberry Pi and Android phone) and present an experimental evaluation that demonstrates that R-Pulsar can enable timely data analytics by effectively leveraging edge and cloud resources.