Analysis on the Relationship between Convolutional Neural Network Layers and Model Degradation

The solution space of the deeper network structure includes the solution space of the shallow network structure, which means the deeper network structure shall get better solutions, and the performance shall also be better than the shallow network. The deep network may be worse than the shallow network either in training or test errors because of degradation. This paper starts with a set of experiments with convolutional neural networks (CNN) trained on pre-trained word vectors for a regression task to assess the Humor in Edited News Headlines. Our research aims to find the influence of the number of convolutional layers on the convolutional neural network and find a model performance best to solve the task. Try to tunning hyperparameters first and then modify the number of layers. The result is that different layers give different results, and it is not the most complex model that produces the best results. When the number of model layers increases, it encounter a degradation problem. Previous research has focused more on solving degradation. Our research is more inclined to explore the regular degradation pattern and find the appropriate number of layers. The final experimental results show that the gradient decline with the increase of the number of layers and time is not very good, and the phenomenon of degradation is encountered. The experiment found the regular pattern of the number of layers and degradation to a certain extent.