Analysis on the Status of Big Data Processing Framework

In the era of big data, various forms of massive data are generated, and the way of dealing with them is also diversified. Common data processing methods are batch processing, stream processing and hybrid processing. In the face of multitudinous data processing framework, it is of practical significance to summarize the types and characteristics of data processing framework and study how to choose the appropriate framework. This paper first introduces several mainstream data processing frameworks (Batch Processing Framework, Stream Processing Framework and Multi Framework Hybrid Processing System), and it briefly describes their related concepts and system architecture. Then it compares and selects typical stream processing framework and multi framework hybrid processing system classification, and gives suggestions on how to choose the appropriate framework. Finally it summarizes and looks forward to the development prospects of big data processing technology.