Incremental Learning of SVM Using Backward Elimination and Forward Selection of Support Vectors

Traditional pattern matching and classification algorithms in machine learning field build models by extracting patterns from whole data at once. It means once a model is learned about the data, the parameters of the model will never be updated and the model remains unchanged forever. However, in the industrial applications, a large volume of data is accumulated incrementally and the dynamics of the data may change over time. The traditional approach of learning the model using all the data at once may not be feasible because it may require a huge storage for data and take a very long time to build a model from a huge volume of data, and the one-time built model may not be able to learn the changed patterns automatically over time. To overcome this limitation of the traditional approach, we propose two novel algorithms of pattern matching and classification, based on Incremental Learning Support Vector Machine (ILSVM), which learn and update models as new data arrives. Our proposed algorithms work based on selection and elimination of support vectors while learning. By performing experiments on 13 publicly available data sets, we found that our proposed methods obtain better accuracy, are computationally faster and require lower memory than the traditional batch learning and ILSVM.