Making the Lipschitz Classifier Practical via Semi-infinite Programming

This paper presents a new implementable algorithm for solving the Lipschitz classifier that is a generalization of the maximum margin concept from Hilbert to Banach spaces. In contrast to the support vector machine approach, our algorithm is free to use any finite family of continuously differentiable functions which linearly compose the decision function. Nevertheless, robustness properties are maintained due to a maximizing margin. To obtain a useful algorithm, the inherent difficult problem is formulated in a convex semi-infinite program. Using this new formulation, we develop a duality result enabling us to solve the original problem iteratively as a finite sequence of constrained quadratic programming problems over a convex hull of matrices. We compare the performance of the Lipschitz classifier algorithm with state-of-the-art machine learning methodologies using a benchmark data set as well as a data set randomly generated from Gaussian mixtures.