Optimizing Real-Time E-Commerce Data Streams: A Comparative Analysis of Serialization and Topic Design Patterns in Apache Kafka

As online shopping grows in popularity, electronic commerce has become an essential aspect of people’s daily lives. The volume of data generated by customers’ interactions with e-commerce platforms has significantly increased. Consequently, analyzing real-time data has become increasingly crucial. In this paper, we propose a set of performance metrics for analyzing real-time data streaming from e-commerce using Apache Kafka pipelines, considering several design considerations such as serialization selection and topic design patterns.