Accumulating Curriculum Knowledge Based on Neural Network Isolated Algorithm

Feeding the training data to neural networks in a specific sequence, from the simple data to the difficult data, utilizing curriculum learning can enhance performance improvements over the typical learning strategy based on shuffling training samples randomly, without any additional computational costs. This training approach has been successfully applied in all fields of machine learning. However, the current curriculum learning will gradually introduce difficult samples until the model eventually performs joint training. In addition, our experimental results find and demonstrate how the order of training tasks arranged according to the difficulty of different tasks influences the reuse percentage of model capacity, allowing us to observe the impact of curriculum learning on model performance from different perspectives. Finally, we also separately conduct three ablations analyses of our model to comprehensively enhance the understanding of the impacts of the proposed approach and further demonstrate its scalability to a standard large-scale dataset, i.e., the ImageNet dataset.