Ermis: a middleware for bridging data collection and data processing in IoT streaming applications

Modern streaming analytics platforms, such as Apache Flink, can receive, process, and act on real-time streaming data from various sources to take immediate action as events occur in the real world. Combined with the emergence of the Internet of Things (IoT), these platforms have revolutionized many application domains by deriving strategic insights from incoming sensor data. For instance, manufacturers can retrofit production lines with sensors, enabling them to rapidly detect and correct problems by analyzing incoming sensor data in real-time. However, when sensors generate data too slowly, the application produces incorrect results due to insufficient information. Conversely, generating data too fast rapidly decreases sensor battery lifetime and wastes storage resources. Consequently, the data generation rate of the IoT infrastructure must match the data processing rate of the application(s) at all times, which is a challenging task due to the several facets of IoT dynamism: sensor mobility, sensor failure, changing application data requirements, varying message inter-generation and transmission delay. Currently, application developers are required to manually configure sensor sampling periods to match these rates; a time-consuming process that is inefficient, if not ineffective, due to the dynamism of the IoT infrastructure. In this paper, we tackle this problem by introducing Ermis, a novel middleware that runs on the gateway and automatically adapts sensor sampling periods at runtime to match application requirements, without relying on any previous knowledge about the underlying IoT infrastructure. To the best of our knowledge, this is the first solution that can ensure the presence of sufficient data for the IoT applications under various sources of dynamism, while minimizing unnecessary transmissions to extend sensor battery lifetime. Extensive evaluation on real-time data from a physical testbed demonstrated a ~34% increase in the expected battery lifetime, while consistently ensuring data sufficiency, compared to state-of-the-art.