Lightweight Data Storage and Caching Solution for MQTT Broker on Edge - A Case Study with SQLite and Redis

In the ever-evolving landscape of edge computing environments, the demand for robust historical data storage and caching solutions has emerged as a critical requirement across diverse scenarios, spanning from mitigating the impact of network outages to facilitating AI/ML applications and accommodating the dynamic movement of edge devices. This technical paper presents a comprehensive case study focusing on the design and implementation of a lightweight, yet efficient data storage and caching solution tailored specifically for the MQTT Broker within the realm of edge computing. Within the Siemens Industrial Edge platform, the Eclipse Mosquitto broker, known as Databus, lacks inherent support for historical data storage and caching mechanisms. To bridge this gap, a meticulous evaluation of various databases was conducted, culminating in the selection of SQLite for data storage and Redis for in-memory caching. The study meticulously delineates the architectural decisions guiding the optimization of this solution, including strategies such as time-series storage, periodic disk writes, temporary buffering, and the provision of configurable storage and caching options. By facilitating swift and resource-efficient data retrieval, the developed solution is tailored to cater to the stringent requirements of low-power edge devices operating within embedded system environments, ensuring optimal performance and scalability.