Parallel, self-organizing, hierarchical neural networks with competitive learning and safe rejection schemes

A new neural network learning algorithm with competitive learning and multiple safe rejection schemes is proposed in the context of parallel, self-organizing, hierarchical neural networks (PSHNN). After reference vectors are computed using competitive learning in a stage of PSHNN, safe rejection schemes are constructed for reference vectors. The purpose of safe rejection schemes is to reject the input vectors which are hard to classify. The next stage neural network is trained with the nonlinearly transformed values of only those training vectors that were rejected in the previous stage neural network Two different kinds of safe rejection schemes, RADPN and RAD, are developed and used together. Experimental results comparing the performance of the proposed algorithms with those of backpropagation and the PSHNN with the delta rule learning algorithm are discussed. The proposed learning network produced higher classification accuracy and much faster learning. The classification accuracies of two methods for learning the reference vectors were compared. When the reference vectors are computed separately for each class (Method II), higher classification accuracy was obtained as compared to the method in which the reference vectors are computed together for all the classes (Method I). This conclusion has to do with rejection of hard vectors, and is the opposite of what is normally expected. In addition, Method II has the advantage of parallelism by which the reference vectors for all the classes can be computed simultaneously