Time Series Prediction of Battery EOL with Intermediate State Knowledge Distillation

Knowledge Distillation (KD) is well known as one of the methods of compressing deep learning models. It aims to train the student model so that the student model (compressed model) can imitate the output of the teacher model (original model). However, the output of the teacher model alone may not be sufficient to train the student model. To compensate for this, we propose a method called Intermediate-State Knowledge Distillation (ISKD). Unlike most of the knowledge distillation studies that have been conducted for computer vision tasks, the proposed method is applied to a time series model that predicts the life of a battery using NASA AMES dataset. As a result, our method reduced the error level of battery End-of-Life (EOL) cycle by up to 1.04% compared to the conventional distillation method. In addition, the number of parameters could be reduced and at the same time, there was an advantage in terms of latency.