Processing math: 12%
--Sigmoid Activation-Based Long Short-Term Memory for Time-Series Data Classification | IEEE Journals & Magazine | IEEE Xplore

\log-Sigmoid Activation-Based Long Short-Term Memory for Time-Series Data Classification


Impact Statement:The proposed activation functions introduce additional hyperparameters in the LSTM-based deep learning model through the use of log-base values. Adding customizability to...Show More

Abstract:

With the enhanced usage of artificial-intelligence-driven applications, the researchers often face challenges in improving the accuracy of data classification models, whi...Show More
Impact Statement:
The proposed activation functions introduce additional hyperparameters in the LSTM-based deep learning model through the use of log-base values. Adding customizability to the activation functions enables the deep learning researchers to better tune their models. The flexibility of the proposed activations unlike the traditional activation functions can play a role in enhancing the performance of LSTM models on time-series datasets.

Abstract:

With the enhanced usage of artificial-intelligence-driven applications, the researchers often face challenges in improving the accuracy of data classification models, while trading off the complexity. In this article, we address the classification of time-series data using the long short-term memory (LSTM) network while focusing on the activation functions. While the existing activation functions, such as sigmoid and \tanh, are used as LSTM internal activations, the customizability of these activations stays limited. This motivates us to propose a new family of activation functions, called \log-sigmoid, inside the LSTM cell for time-series data classification and analyze its properties. We also present the use of a linear transformation (e.g., \log \tanh) of the proposed \log-sigmoid activation as a replacement of the traditional \tanh function in the LSTM cell. Both the cell activation and recurrent activation functions inside the LSTM cell are modified with \log-sigmoid activation family while tuning the \log bases. Furthermore, we report a comparative performance analysis of the LSTM model using the proposed and the state-of-the-art activation functions on multiple public time-series databases.
Published in: IEEE Transactions on Artificial Intelligence ( Volume: 5, Issue: 2, February 2024)
Page(s): 672 - 683
Date of Publication: 07 April 2023
Electronic ISSN: 2691-4581

Funding Agency:


References

References is not available for this document.