Research on hybrid collaborative filtering recommender system based on spark

Based on the study of the recommender engine and big data process platform, a hybrid collaborative filtering recommender method is proposed to solve the drawbacks of the current recommender algorithm in practical engineering application. Dimensionality reduction and clustering are adopted to overcome the limitations of collaborative filtering such as sparsity of data. Apache Spark is used to realize an efficient parallel implementation. With the increase of data, the scalability of recommender scheme can be solved by expanding Spark cluster nodes. The advantages of the proposed method have been proven in the practical application of the teaching information resource service platform of a university.