Discovering Contrast Sets for Efficient Classification of Big Data

Due to the recent emergence of Big Data, it is essential to develop techniques to reduce the processing time of such Big Data. In this paper, we propose to reduce the dimensionality of objects' feature vectors by discovering their contrast sets. Contrast set mining aims at finding a set of rules that best distinguish the instances of different user-defined groups. Thus, contrast sets are conjunctions of attribute-value pairs that are significantly more frequent in one group than the other. Existing techniques extract contrast sets from categorical data or discretized numerical data. Furthermore, existing rule-based contrast sets methods require some user-defined thresholds to select the contrast sets. To overcome these limitations, we propose a greedy algorithm, called DisCoSet, to incrementally find a minimum set of local features that best distinguishes a class from other classes without resorting to discretization. We show that the proposed algorithm reduces the dimensionality between 40%-97% of the original length and yet improves the classification accuracy by 10%-24% on different datasets.