A support vector machine approach to decision trees

Key ideas from statistical learning theory and support vector machines are generalized to decision trees. A support vector machine is used for each decision in the tree. The "optimal" decision tree is characterized, and both a primal and dual space formulation for constructing the tree are proposed. The result is a method for generating logically simple decision trees with multivariate linear, nonlinear or linear decisions. By varying the kernel function used, the decisions may consist of linear threshold units, polynomials, sigmoidal neural networks, or radial basis function networks. The preliminary results indicate that the method produces simple trees that generalize well with respect to other decision tree algorithms and single support vector machines.