The Task of Identifying Morphological Errors of Words in the Kazakh Language in Social Networks

On the Internet and on Instagram, Vkontakte, Twitter and other social networks, applications are very attractive in terms of receiving and analyzing information in messages, because the information in these systems is real. However, text posts, user comments in social. networks often differs from the generally accepted norms of the language. There are mistakes in it, deliberate distortions of words. Unfortunately, such inaccurate data cannot be taken into account in content analysis, statistical or sentiment analysis of data. Words with such errors, i.e. incorrect words can be processed and analyzed, identifying and correcting them into the correct form. This paper presents research and implementation of a model for determining incorrect words of the Kazakh language in semi-structured data, using the example of comments and posts on social networks. To solve and analyze the task, the following was done: a comparative analysis of text correction systems was carried out; explored various spell-checking technologies; the most frequently used errors are identified and the classification of errors in words is presented. A dictionary of the basics of the Kazakh language from the electronic corpus has been developed. The authors have developed an approach to identify incorrect words and auto-replace with a suitable candidate object (word) for the Kazakh language. To solve the problem, a approach is presented to detect incorrect words in the Kazakh language. An approach has been developed for identifying incorrect words from semi-structured data, which is based on the stemming algorithm with lexicon stems according to the CSE (Complete Set of Endings) Experimental calculations and evaluation of the results have been carried out.