A relevancy filter for constructive induction

Some machine-learning algorithms enable the learner to extend its vocabulary with new terms if, for a given a set of training examples, the learner's vocabulary is too restricted to solve the learning task. We propose a filter, called the Reduce algorithm, that selects potentially relevant terms from the set of constructed terms and eliminates terms that are irrelevant for the learning task. Restricting constructive induction (or predicate invention) to relevant terms allows a much larger explored space of constructed terms. The elimination of irrelevant terms is especially well-suited for learners of large time or space complexity, such as genetic algorithms and artificial neural networks. To illustrate our approach to feature construction and irrelevant feature elimination, we applied our proposed relevancy filter to the 20- and 24-train East-West Challenge problems. The experiments show that the performance of a hybrid genetic algorithm, RL-ICET (Relational Learning with ICET), improved significantly when we applied the relevancy filter while pre-processing the data set.