Hyperparameter Optimization for Deep Residual Learning in Image Classification

Convolutional Neural Network (CNN) is one of the most commonly used deep learning models to train a large number of datasets and getting valuable results in image recognition. Deep Residual Learning (ResNet) is one of the most famous CNNs for the computer vision tasks that won the ILSVR-2015 classification competition. ResNet is also one of the deepest models to train the neural networks with the idea of identity mapping for short connections. In this paper, we built a hyperparameter based approach for ResNet architecture and show that our optimization leads to significant performance improvements. We have built a manual search approach by altering the traditional data augmentation proposed by the previous approaches. Our manual approach significantly reduces the computing time compared with the previous automatic approaches. Experimental results show that our approach significantly improves the classification error rate of the ResNet for the CIFAR-10 dataset. We achieved a 5.20% error rate for ResNet-164 with the significantly reduced computing time.