Distributed Music Comments Mining System Based on BGLL Algorithm

This paper aims to establish a distributed music comments mining system based on BGLL algorithm. With the massive music resource and users of NetEase Cloud Music, our system employs big data technology and machine learning technology to analyze data from the perspective of comments, discover users' attitudes towards the song or the music. The whole system is built based on the Hadoop ecosystem, and it includes four main modules: data acquisition, data storage, data analysis, and data visualization. A python based web crawler is used for data acquisition on NetEase Cloud Music resource, then the distributed HDFS system is built for data processing and analyzing. A refined mining algorithm based on BGLL is implemented for topic mining procedure. At last, a word cloud map is generated based on javaweb technology. Experiments show that our system can perfome well on the comment topic mining application, due to the distributed architecture, our system has a potential capability on dealing with large scale of data.