Designing of a Classifier for the Unstructured Text Formalization Model Based on Word Embedding

The active use of artificial intelligence technologies has a direct positive impact on the development of society in various areas of human life. The article describes the developed model of processing and formalization of textual unstructured information in the form of a continuous flow of text information taken from the news feed of news agencies. A method for preprocessing text to reduce the execution time of the algorithm and save CPU resources is given. A method for representing words as a real vector is formed using various algorithms for training artificial neural networks and their properties. A model of the first stage of the text information processing system as a subsystem for classifying the subject of a news article text based on a vector representation of words, including a description of the word vectorization algorithm, an example of the type of word structure with a corresponding numeric vector, and a metric that determines the proximity of vectors to each other in space. The results of the experiment are obtained and a method for setting a decision criterion for the implemented classifier is proposed. The area of use of the proposed classifier is the sphere of information security. The results of the experiment can be indicators of the suitability of using the classifier as a definition of the subject of a news article.