Analysis Range of Coefficients in Learning Rate Methods of Convolution Neural Network

Convolutional Neural Network (CNN) is a type of feed-forward artificial neural network, exploiting the unknown structure in input distribution to discover good representations with multiple layers of small neuron collections. CNN uses relatively little pre-processing compared to other classification algorithms, usually uses gradient decent to updates the parameters in the network. Since CNN was introduced in 1997s to deal with face recognition, it has made much achievement in many fields, and has been the state-of-the-art method in face recognition, speech recognition, etc. To get small error rate and a better speed of training the CNNs, a lot of learning rate methods are proposed. In this paper, we analyzed the range of the coefficients in these methods with a restriction of max convergence constant step learning rate. In our experiments, we find the max convergence learning rate by dichotomy with little computation cost, we also confirm the range of coefficients is useful. Moreover, we gives a comparison among these methods on speed and error rate.