Web Scraping for Summarization of Freelance Job Website Using Vector Space Model

In the contemporary digital age, the internet plays a central role in various community activities, much like it does in the realm of work. Nowadays, numerous platforms offer job listings, particularly geared toward freelancers. To access this information, users typically have to visit multiple websites to find relevant job opportunities. Web scraping presents a solution to address this challenge. Building upon prior research, we will utilize the BeautifulSoup and Selenium libraries for data collection. Our data retrieval will rely on the vector space model method to assess the similarity between the user's query and the documents. In our data examination, we achieve a perfect recall rate of 100%, but the average precision rate is at 56%. This is explained by the fact that our data search uses three factors, increasing the possibility that when a document contains a word from the user's query, even if it doesn't exactly fit the context, unrelated material will be retrieved. We can show the outcomes of data processing and walk people through the site scraping, data processing, and data retrieval processes by utilizing the Python Streamlit framework. This study's main objective is to use web scraping techniques to get information from freelancing websites. Users may get data from various websites by using the vector space model technique, eliminating the need to visit each freelancing site separately. Furthermore, by employing data visualization in a web application developed with the Streamlit framework, we can present the web scraping results in a more user-friendly format, ultimately saving users valuable time.