Deep convolutional network neocognitron: Improved Interpolating-Vector

The neocognitron is a multi-layered convolutional network that can be trained to recognize visual patterns robustly. In the intermediate layers of the neocognitron, local features are extracted from input patterns. In the highest (or deepest) layers of the network, the method of Interpolating-Vector is used for classifying patterns based on the features extracted by the intermediate layers. During the learning, several reference vectors for each class are created from a set of training vectors. To recognize an input vector, we measure distances (based on similarities) between the input vector and planes that are spanned by every trio of reference vectors of the same class. The class name of the nearest plane is taken as the result of classification. To reduce the computational cost, we propose to search the nearest plane, not among all possible combinations of three reference vectors, but only among trios that contain the nearest reference vector. For reducing the computational cost, it is also important to represent the large number of training vectors accurately with a compact set of reference vectors. To create a compact set of reference vectors, the learning is carried out in two steps. In the first step, reference vectors are just chosen from vectors in the training set. We start modifying reference vectors (namely, fine tuning of connections) from the second step after an enough number of reference vectors have been chosen. The effectiveness of the proposed method for recognizing hand-written digits is demonstrated by computer simulation.