A Sequence Model Compression Method Based on Proper Orthogonal Decomposition

We propose a method that can compress the parameters and floating point operations (FLOPs) of a Long Short-Term Memory (LSTM) model to a third of the original model and get an acceptable error without fine-tuning. The LSTM can be treated as a discrete nonlinear dynamical system and use a kind of model order reduction method which is named proper orthogonal decomposition (POD) to deal with it. Approximating the hidden state reduces the dimensionality of the hidden state. We design a small LSTM model that can be considered as a time series classification model and a public dataset to validate the method. The number of parameters of the original model can be compressed to 34.4% and the FLOPs can be reduced to 25.6% with a 5% loss in accuracy.