An algorithm for automatic design of two hidden layered artificial neural networks

This paper presents an algorithm that automatically designs compact two hidden layered artificial neural networks (ANNs). The algorithm, which we call the cascade neural network design algorithm (CNNDA), starts network building process in a constructive fashion by adding one node at a time. Once the network converges CNNDA starts pruning of the networks by deleting hidden nodes and connections sequentially. The pruning process is repeated until the network no longer converges. In order to improve the computational efficiency and convergence, CNNDA adopts a training process which temporarily freezes the input weights of a hidden node when the output of that node does not change much in the successive few iterations. However, the freezed weights may unfreeze in the pruning process for further modification. CNNDA has been tested on three benchmark, i.e., character recognition, cancer and diabetes, problems. The experimental results show that CNNDA can produce very compact ANNs with a small classification error in comparison with other algorithms. It is seen that there is a tradeoff between compact ANN size and training epochs.