Development of the Concept and Architecture of an Automated System for Updating Physical Knowledge for Information Support of Search Design

For developing an automated system for updating physical knowledge for information support of search design, it is necessary to choose a technology stack that would meet the implementation requirements. In view of the sanctions currently imposed on the Russian Federation, it is worth considering mainly open projects and/or domestic developments. We will highlight the main criteria that it is desirable to take into account when designing the architecture of an automated system to support the synthesis of new technical systems and technologies: (a) ability to store and process large amounts of data; (b) unification access for all data analysis procedures; (c) maximum automation of all stages; (d) modularity of the structure, focus on the expansion of functionality; (e) focus on open source solutions and software of domestic manufacturers, excluding rigid binding to paid foreign solutions. As a result of the work done, various aspects of the implementation of the required automated system were analyzed. A review of various software systems and cloud products showed that the concept of building data lakes (Data Lake) in conjunction with the distributed processing tools of the Apache Hadoop ecosystem is used for big data processing. An architecture framework based on a centralized data warehouse and Hadoop components is proposed. It will be possible to increase the functionality of the platform by adding new microservices that connect to the storage and distributed processing tools via the API, as well as using a single web service for managing and displaying data analysis results from these microservices.