Improving Generalization in Convolutional Neural Networks with a Dynamic Attention Layer

This paper addresses the challenge of out-ofdistribution (OOD) learning in computer vision by introducing a novel Dynamic Attention Layer (DAL), designed to enhance Convolutional Neural Networks. DAL dynamically interchanges attention weights, based on selected percentiles during training, improving the network's ability to capture both dominant and secondary features of the data. We assessed DAL in three scenarios: (1) augmented MNIST images to evaluate resilience to transformations; (2)a novel dataset of unseen image classes to test performance on new instances; and (3) a DALL-E generated dataset to challenge class differentiation with mixed features. Our results show that DAL enhances accuracy and reduces loss in both in-distribution and OOD scenarios. The optimal configuration of parameters for DAL was found to maximize model performance, demonstrating its potential to address OOD learning challenges and contribute to more robust computer vision systems. Keywords-attention layer, convolutional neural networks, out-of-distribution learning, feature acquisition, deep learning