A Novel Solution to the Curse of Dimensionality in Using KNNs for Image Classification

The k-Nearest Neighbors (KNN) is one of the simplest and widely used algorithms in Machine Learning applications such as Image Classification. Being based on the Euclidean distance the algorithm is quite simple and effective in most cases. However, it suffers from the problem of ""The Curse of Dimensionality"" as the Euclidean distance becomes meaningless when the dimension of data becomes significantly high. In this paper we present a novel solution to this problem by making use of the Convolutional Neural Network (CNN) which can extract the most important features automatically from the images. These features extracted by the CNN are of reduced dimensions and can effectively be used by the KNN to recognize the images. The results and comparisons show that the proposed method is also seen to reduce the time taken for testing while retaining high accuracy. The proposed technique achieved an accuracy of 96.92% on MNIST, 85.09% on Fashion MNIST and 95.17% on the A-Z Alphabets databases respectively.