Second order design of multiclass kernel machines

In order to reduce the computational complexity of kernel machines and improve their performance in multi-label classification, we develop a systematic two step batch approach for constructing and training a new multiclass kernel machine (MKM). The proposed paradigm prunes the kernels, and uses Newton's method to improve the kernel parameters. In each iteration, output weights are found using orthogonal least squares. Algorithm performance is compared to that of least square support vector machines and support vector machines. Simulation results on many benchmark and real life datasets show that the proposed algorithm results in smaller network size and better generalization.