Non-Myopic Feature Selection Method for Continuous Attributes and Discrete Class

Currently there exist diverse feature selection ranking methods and metrics for databases with pure discrete data (attributes and class), or pure continuous data. However, little work has been done for the case of continuous attributes with discrete class, and at the same time evaluating attribute subsets in a non-myopic fashion, considering its inter-dependencies or interactions. Normally what we can do is perform discretization, and then apply some traditional feature selection method; nevertheless the results vary depending on the discretization method that we utilized. Additionally, if we only evaluate isolated attributes, we probably obtain poor results, because we are not considering attribute inter-dependencies. We propose a metric and method for feature selection on continuous data with discrete class, inspired in the Shannon's entropy and the information gain, which overcomes the above problems. In the experiments that we realized, with synthetic and real databases, the proposed method has shown to be fast and produce near optimum solutions, selecting few attributes.