On Exploring Non-negative Matrix Factorization for Deep Neural Network Compression

Deep Neural Networks (DNNs) are pivotal across diverse domains, yet over-parameterization hampers the efficiency of software implementations. Hardware implementation offers faster inference and lower power consumption. However, optimizing architectures remains challenging. To address these challenges, we propose a novel Non-negative Matrix Factorization (NMF) based method for matrix decomposition with sparse pruning. It has been shown that the proposed method achieves a maximum compression rate of 80.76% in pruning the AlexNet model trained on the MNIST dataset while keeping the original test accuracy intact.