Fusion of privileged features for efficient classifier training

Within the supervised machine learning framework, classifier performance is significantly affected by the size of training datasets. One of the ways to improve classification accuracy with small training datasets is to utilize additional knowledge about training data that is not present in testing data. In the Learning Using Privileged Information (LUPI) learning paradigm, this additional knowledge is represented by the privileged feature space, which is fused with the standard feature space during classifier training. In this paper, we study two new algorithmic realization of LUPI, based on knowledge transfer, and compare them with the original SVM+ LUPI algorithm, which is based on similarity control. We apply the algorithms to a wide area aerial video exploitation problem, and measure misclassification error rate and execution time of both algorithms. We use minor area motion imagery (MAMI) dataset, recently released by Air Force Research Laboratory (AFRL). High-resolution imagery is utilized as privileged information for low-resolution imagery. We demonstrate that an ensemble knowledge transfer approach that combines multiple knowledge transfer algorithms outperforms the original SVM+ approach with the error rate reduction of up to 25%. We also show that knowledge transfer approach requires significantly less computational resources, making LUPI algorithms as scalable as SVM.