A Contemporary Research Study on Web Scraping and Innovation

The digital transformation of manufacturers is measured through the selection of transformation technology, the designed scope of work, and the subsequent results of the digital implementation. The increased use of digital technology in the manufacturing sector has produced positive results, such as fostering corporate innovation, reducing trade costs, optimizing resource allocation, optimizing organizational efficiencies and improving financial performance. Indirectly, digital transformation can reduce an organization's carbon emissions [3]. At the micro-level of enterprises, a web scraping project was utilized as an accessible and inexpensive method of accessing web data to assess the digital maturity from a sample of Minnesota and North Dakota manufacturers that had been presented with surveys assessing sociotechnical readiness [13]. The keywords examined represent the nine fields of Industry 4.0, which are cyber- physical systems, the internet of things, big data, cybersecurity, cloud computing, additive manufacturing, advanced robotics, modelling and simulation, and augmented virtual reality. Through the methodology of web data extraction, this study reviewed a sample population of 149 websites of regional manufacturers. The data was reviewed through text analysis techniques to ensure the relevancy of the keywords to the project subject matter.