Classification using Perceptron in Low Feature Space

A perceptron can handle binary problems and can classify linearly separable data only. Solving a linearly non-separable problem with a perceptron generates numerous decision boundaries, and uses a large number of iterations for adjusting weights and bias. Such classification becomes more complicated if the problem belongs to high-dimensional feature space. Thence, either Kernel Functions, additional hidden layers, or modified activation functions are introduced. Since, in low feature space, a perceptron can proficiently handle a non-linear problem without adding layers or using kernel methods. Therefore, the proposed effort suggests the concept of shifting of origin for solving such problems in low feature space. The work also uses the properties of Euclidean distance, a mid-point, and the lines separating the two classes, to identify the efficient perceptron. As obvious, a straight decision boundary can sufficiently distinguish linear data, while a bunch of lines is still not efficacious with non-linear data. Hence, the suggested approach moves the existing origin to different points in the low feature space (or xy-plane), such that each time the origin is shifted, the plane is further divided into four more quadrants. Thus, the entire plane is divided into the smallest possible quadrants comprising multiple lines. Lines possessing a classification accuracy above 80% are selected to form a convex hull (or required perceptron) whose shape depends on the data. It means three lines form a triangular convex hull, eight envision an octagonal convex hull, and so on. Moreover, the one-vs-one approach is used to address the problem of multi-class classification. Finally, the experimental analysis revealed that using only two optimal features, the proposed approach can effectively handle both linearly separable and non-separable problems without increasing the complexity of the problem, which is also evident from the high accuracy rates achieved during classification.