Large margin classifier via semiparametric inference

In this paper, we construct a learning method of stochastic perceptron based on semiparametric inference, and show that this method produces large margin solutions. In semiparametric inference, the parameters are divided into structural parameters which are to be estimated and nuisance parameters in which we do not have any interest. Here, the weight vector of perceptron is defined as structural parameters and the steepness of transfer function is defined as a nuisance parameter. Usually, rough estimate is substituted to nuisance parameters and only structural parameters are estimated. To compensate the estimation error caused by rough estimate, an additional term is added to the derivative of likelihood. We will show that this additional term is related to the regularization term which causes large margin solutions. This work suggests that the success of large margin classifiers can be attributed to semiparametric inference.