Novel Application of MapReduce and Conceptual Graphs

Knowledge discovery is the process of extracting useful or hidden patterns in data. With the growth of data in a structural form, such as social networks, extracting knowledge from data represented in the form of graphs is an emerging technique. In this paper, we demonstrate how "skills" data from resumes (i.e., what skills an applicant possesses) can be modelled into a type of graph data structure called a conceptual graph using the MapReduce programming model. Initial storage and pre-processing is done in a big data framework using the well known Hadoop Distributed File System (HDFS) and MapReduce, and skill-set discovery is accomplished using well-established graph mining techniques. We empirically evaluate our approach in the domain of skill-set analytics, where common skill-sets are extracted from a dataset of resumes. Common skill-set extraction is useful for course curriculum designers as well as job seekers.