Technologies of distributed data stream processing based on big data

In the big data environment, data stream processing requires high real-time performance, and data calculation requires persistence and high reliability. Distributed Data Stream Processing System (DDSPS) can solve the problem of data stream processing in the big data environment. In addition to the advantages of scalability and fault tolerance of distributed systems, it also has high real-time processing capabilities. This article introduces three open source distributed streaming data processing systems, and compares and analyzes these three streaming frameworks. The research content can provide technical reference for the theoretical research and application technology development of data stream processing in the big data environment.