The Science of Rule-based Classifiers

The exponential rise in the amount of data has fueled various facets in data science and data engineering. Classification is a supervised machine learning task that underlies several diagnosis systems and rule-based decision support systems. The efficiency of a rule-based classifier depends on factors such as quality of the rules in the ruleset, rule ordering, and cardinality of the ruleset. This paper experimentally analyzes the effect of case satisfaction mechanisms and rule selection approaches in the design of rule-based classifiers with a focus on clinical datasets. Moreover, a hybrid rule combination approach called rule pool is proposed. Rule pool combines the best rules from a decision tree classifier and associative classifiers. Results of the experiments with 5 medical datasets obtained from the University of California Irvine repository show that the proposed hybrid approach has its advantages in comparison with previous methods in terms of classification accuracy. Rules from Decision trees on an average yielded an accuracy of 79.92%; Associative Classifiers yielded an accuracy of 74.79% and the hybrid rule pool based approach yielded an accuracy of 80.95%. Moreover, a difference in classification efficiency based on the Best-first and Best-fit case satisfaction methods was inferred; the former yielded an average accuracy of 76.89% and the latter yielded 79.87%. From the analysis of the experimental results, it was inferred that the rule pool includes informative rules from decision trees and also the top-ranked rules of the associative classifiers. This causes the efficiency of the rule pool approach to be relatively higher than the traditional rule-based classifiers and associative classifiers. Moreover, on combining the rules from different algorithms, when the one-item rules were replaced by longer non-redundant strong rules, an improvement in accuracy was observed. The findings of this work may be considered for designing efficient rule-based classification approaches for clinical knowledge-mining framework.