Stateful Stream Processing for Digital Twins: Microservice-Based Kafka Stream DSL

Digital Twin is a virtual representation of a technological process or a piece of equipment, that supports monitoring, control and state prediction based on the data, gathered from the sensor networks. To parallelize event processing and produce near-real-time insights over data streams, Digital Twin should be implemented based on an Event-Driven architecture. The Event-Driven architecture is loosely-coupled by its nature. One of the recent possible solutions for loose coupling system is a Microservice approach, a cohesive and independent process that interacts using messages. Stateless behavior is the nature of the microservice, but on the other hand, the vast majority of stream processing in Digital Twin imply stateful operations. Thus, in this paper, we propose a case-study of the possibility to use Apache Kafka Stream API (Kafka stream DSL) to build stateful microservice for real-time manufacturing data analysis. Also, in the presented work we discuss the fulfillment of such requirements as fault tolerance, processing latency, and scalability to support the stateful stream processing in Digital Twins implementation.