Detecting Scrapers on E-commerce Websites Using a Reduced Feature Set

The amount of traffic on the Internet originated by web bots has been on the rise for many years. The use of bots for illicit web content retrieval and functional abuse is rinsing high concerns across website operators which are constantly battling these malicious entities capable of bypassing their anti-bot protection systems. In this work, we study bot behavior inside modern e-commerce websites within the travel industry. After an initial analysis of bot characteristics, we describe a pipeline capable of autonomously detecting sophisticated scrapers. We observed that these scrapers can be easily identified using a reduced set of features designed for an efficient and interpretable detector.