Machine learning techniques in Hadoop environment: A survey

A plenty of informative data is produced on social sites. Twitter is a widely used social site to express opinion and feelings related to any domain. Tweets can be used to extract relevant and reliable information. Data analysis is an important task for mining these types of large data. The classification of these tweets can be useful to human beings. However, storage and processing of plenty of tweets is a challenging problem. Hadoop Distributed File System (HDFS) can be used to store this data. To extract information and to classify tweets, various data mining techniques can be utilized. The well-known data mining techniques include Clustering and Classification. Traditionally, these algorithms run in the sequential environment. For improving computational performance and address scalability issue, these methods should run in distributed environment. However, distributed environment introduces issues like synchronization, communication overhead, appropriate division of tasks/data, and load imbalance. The problem of synchronization can be handled using Hadoop Map-reduce programming model. This paper aims to discuss parallelism of various data mining algorithms in Hadoop environment.