Optimization of one and two hidden layer neural network architectures for handwritten digit recognition

This paper presents extensive results of training architectures of one- and two- hidden-layer, fully connected, feedforward neural networks which are used for handwritten digit recognition. Both architectures include 1 to 400 units in each hidden layer to produce a series of 112 networks. These networks are trained by backpropagation up to 1000 iterations and tested while advancing in training. Results are presented as a function of the number of hidden units, the number of iterations, and the training mode (centered and position-shifted patterns). An overall maximum recognition rate of 92.4% is obtained for a two-hidden layer network with 100 hidden units and 144 iterations. No significative recognition rate improvement is achieved (within 2.1 percentage points) by increasing the number of hidden units and iterations above 50 and 30, respectively.