Analysis of Reviews of Tourist Objects Based on Large Language Models

The article is devoted to solving the problem of identifying issues in the field of regional tourism. To solve this task, the method of aspect-based analysis of reviews of tourist objects using of large language models and prompt-engineering has been developed and implemented. Accommodation facilities, dining establishments, services and attractions are considered as tourist objects. The reviews are obtained from tourist information aggregators using web-scraping method. The initial dataset consisted of 1300 tourist reviews. A large language model, Saiga2, with 70 billion parameters was chosen for the implementation of prompt engineering. The problem analysis task was divided into subtasks of extracting and classifying facts (advantages and disadvantages). For each subtask, prompts have been developed with an indication of the role, a description of classes and examples of solutions. The description of classes is based on the literature analysis of issues in tourism. The method is implemented on the Yandex. Cloud platform, tested and validated to search for accommodation facilities satisfying tourist's requests for Olkhon Island. The results of the method's accuracy evaluation are described using metrics f1, micro f1, macro f1 and weighted f1. To test the proposed method validation, tests describing information about different categories of tourists were developed. The performance of the method was evaluated by comparing the solutions provided by the tourism expert and the created method implementation. The experiment showed 80 percent coincidence of the solutions. The method is planned to be implemented in the service of regional tourism monitoring of the Baikal region.