An Empirical Study on Performance Measures for Online Advertising

In this paper, we study performance measures for online advertising systems by empirical approach. The traditional statistical measure Critical Success Index (CSI), also known as Threat Score (TS), is modified as CSI-R to add the emphasis on recall. We then generate two realistic data sets to examine the effects of CSI and CSI-R in evaluation of a Real-Time Bidding (RTB) system for online advertising. Performance factors such as accuracy, lift, efficiency and skill score are compared for both minority and majority classes. The results show that CSI-R is better applied on measuring such a system.