Alleviating Over-Fitting in Attribute Reduction: An Early Stopping Strategy

In rough set theory, forward heuristic algorithm selects the most important attribute in the process of attribute reduction until the given constraint is satisfied. However, the attributes selected by such strategy may bring us over-fitting. To solve such problem, a new heuristic algorithm is designed: the importance of the attribute is obtained by cross validation and then the Early Stopping is employed to terminate the algorithm if over-fitting occurs. Based on the neighborhood rough set, the heuristic algorithm is compared with the new method over several UCI data sets. The experimental results show that: 1) the proposed algorithm can effectively alleviate over-fitting; 2) the reduct obtained by the new algorithm may offer us better classification performances.