Perceptrons revisited: the addition of a non-monotone recursion greatly enhances their representation and classification properties

In this paper we describe a novel type of adaptive system and compare its representation and classification performances with classical solutions. The main feature of our system is that it is based on combining simple perceptrons with a compact and simple to implement nonlinear transform defined as a finite recursion of simple nonmonotonic functions. When such a nonlinear recursion replaces the standard output function of a perceptron-like structure, the representation capability of Boolean functions enhances beyond that of the standard linear threshold gate and arbitrary Boolean functions can be learned. While the use of nonlinear recursion at the output accounts for compact learning and memorization of arbitrary functions, it was found that good generalization capabilities are obtained when the nonlinear recursion is placed at the inputs. It is thus concluded that the proper addition of a simple nonlinear structure to the well known linear perceptron removes most of its drawbacks, the resulting structure being compact, easy to implement, and functionally equivalent to more sophisticated neural systems.