Adaptation of Transformer Model for Numeric Case

The present paper discusses the adaptation of the transformer model for the case of numerical data. Although transformers are usually associated with large language models, there are many areas where the dependencies between the numeric input and the numeric output are too complex for statistical machine learning techniques. To apply a transformer for the numeric case, alternations of tokenizer and encoder-decoder mechanisms have been made. The proposed technique has been tested to estimate the wave specter of coastal seas on the basis of satellite-based synthetic aperture radar imagery and has demonstrated high accuracy. The mean correlation coefficient of 0.71 was achieved.