Design and Optimization of a Network Live Streaming Recommendation System Based on Collaborative Filtering Algorithm

In order to improve the viewing volume of users and the conversion rate of new users, the author proposes a network live streaming recommendation system based on collaborative filtering algorithm. Firstly, based on Spark, Flink, search engines Elasticsearch, Kafka, data warehouse Hbase, etc., build a user oriented big data recommendation system. Experimental results have shown that the Flink algorithm runs much faster than the Flink algorithm, and its runtime increases with user behavior. However, the runtime of Flink remains almost unchanged, and the gap between the two is gradually widening. At the same time as 80000 words, the processing speed of Spark Streaming has reached 2 seconds. If Spark Streaming and other technologies are used to implement an online recommendation engine, combined with the processing time of other modules, the entire online processing time will far exceed 2 seconds, greatly exceeding the requirements of online processing. On this basis, the author proposes a Flink based recommendation method, which can effectively improve the real-time performance of recommendations. The results indicate that the system is feasible.