AcDWS:Actor Based Distributed Web Scraping

This paper examines a new system for web data extraction that utilizes an AcDWS approach. The main goal of this system is to collect information related to user-requested products quickly and at scale. In this system, communication between actors is managed, and the messages exchanged between them are controlled to ensure proper operation. Solr is used for data storage and full-text search, Akka for parallel processing, and Akka Cluster for distributed processing. The results show that this method operates 75% faster than conventional methods using Python and 36.7% faster than data extraction implemented in Java sequentially.