Empirical Analysis of Impact of HTTP Referer on Malicious Website Behaviour and Delivery

Referer is a HTTP header field transmitted to a webserver, which allows the webserver to identify the origin of the request and the path taken by the visiting user to reach the final resource. Although referer is an optional field within an HTTP protocol header, many webservers use the information for logging, marketing and analytical purposes. Referer has, however, been abused in web spam cloaking and search engine optimization (SEO) attacks. The latter increases a malicious website's ranking in a search engine result with the aims of delivering spam to unwitting users. In this paper, we undertake a quantitative study to determine the effects of referer information on delivery of malicious content (excluding spam) and whether different referer values, mimicking an average user will yield dissimilar results in terms of the number and type of attacks. Our study of 500,000 suspicious websites confirms that similar to web spam, referer information is a HTTP header variable used by malicious websites to distinguish regular users from automated crawlers and security tools, and is abused to deliver malicious content accordingly.