Understanding the Dynamics of the Evolution of Weights in Neural Networks using Dynamic Mode Decomposition Approach

This paper employs Dynamic Mode Decomposition (DMD) and explainable Artificial Intelligence (XAI) techniques to comprehend temporal dynamics in neural network weight convergence. Encompassing four experimental investigations, including proper training, overfitting, and specialized layers like CNNs, RNNs, and LSTMs, the study uncovers spatial weight pat- terns underlying evolving network learning dynamics. The DMD analysis consistently reveals a convergence trend, with weights stabilizing and converging towards specific points on the Argand plane, indicating static behaviors. These outcomes emphasize methodological limitations in capturing dynamic evolution within neural network weights during training. The paper advances a nuanced understanding of the temporal facets intrinsic to neural network training dynamics and underscores the distinct traits associated with various layer types, supported by XAI insights.