Loading web-font TeX/Math/Italic
Fractional Tensor Recurrent Unit (fTRU): A Stable Forecasting Model With Long Memory | IEEE Journals & Magazine | IEEE Xplore

Fractional Tensor Recurrent Unit (fTRU): A Stable Forecasting Model With Long Memory


Abstract:

The tensor recurrent model is a family of nonlinear dynamical systems, of which the recurrence relation consists of a p -fold (called degree- p ) tensor product. Des...Show More

Abstract:

The tensor recurrent model is a family of nonlinear dynamical systems, of which the recurrence relation consists of a p -fold (called degree- p ) tensor product. Despite such models frequently appearing in advanced recurrent neural networks (RNNs), to this date, there are limited studies on their long memory properties and stability in sequence tasks. In this article, we propose a fractional tensor recurrent model, where the tensor degree p is extended from the discrete domain to the continuous domain, so it is effectively learnable from various datasets. Theoretically, we prove that a large degree p is essential to achieve the long memory effect in a tensor recurrent model, yet it could lead to unstable dynamical behaviors. Hence, our new model, named fractional tensor recurrent unit (fTRU), is expected to seek the saddle point between long memory property and model stability during the training. We experimentally show that the proposed model achieves competitive performance with a long memory and stable manners in several forecasting tasks compared to various advanced RNNs.
Published in: IEEE Transactions on Neural Networks and Learning Systems ( Volume: 36, Issue: 2, February 2025)
Page(s): 3598 - 3607
Date of Publication: 15 December 2023

ISSN Information:

PubMed ID: 38100343

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.