Two-phase Switching Optimization Strategy in LSTM Model For Predictive Maintenance

Predictive maintenance has emerged as an important strategy in the industrial operations due to the current advancement on machine learning approaches. The use of Long Short Term Memory (LSTM) neural network can infer sequential data in time domain with an easy deployment for predictive maintenance. An efficient optimization is required in training a LSTM model to update the weights. In this paper, the Stochastic Diagonal Approximate Greatest Descent algorithm (SDAGD) with momentum is proposed to optimize the learning parameters of LSTM cell using a two-phase switching optimization strategy. First, an iterative step of SDAGD algorithm is defined at the boundary of local search region if the local search region does not contain a global solution. This optimizer switches to the Newton method using the controlling parameter of relative step- length when an optimal solution does exist in the search region. The implementation of momentum in the SDAGD algorithm is performed on LSTM model optimization to accelerate the convergence. As a result, this optimizer has reduced both the testing and training error rate with the acceleration aspects of momentum parameter.