Automation of Data Analysis and Web-Scraping to Value Used Items

Second-hand products such as phones, laptops, car’s, etc. make up a substantial part of the market. They aid a sustainable lifestyle by reducing the use of new resources and diminishing waste disposal. With the evolution of second-hand shopping, pricing has become a very tedious and cumbersome process. Traditionally, models are pre-trained on static datasets. With the trends in the market being sporadic, the outcome of such models is greatly affected. One approach to counter this would be to update the dataset periodically, which requires significant resource allocation. This approach is better suited when dealing with a dataset that contains a large range of products together but may prove inefficient when encountering a specific item. Through this study, a pipeline is established that integrates real-time web scraping with machine learning, which is achieved by using the Selenium tool, automating data preparation of the web scraped data, and implementing a machine learning model. The model considers essential item features to provide accurate predictions for specific items queried by a user. This paper also helps us perform a comparative study of various machine learning models that are considered ideal for regression analysis. When dealing with a dataset, limited in size, the need to pick a model that can handle the sparsity in data becomes critical. Tuning the Random Forest model to 30 trees showed the best results when compared to the other regression models. It predicted accurate prices on multiple iterations with varying inputs, whereas the other models did not account for the change in the features.