BCQ4DCA: Budget Constrained Deep Q-Network for Dynamic Campaign Allocation in Computational Advertising

Digital advertising companies typically conduct several advertising campaigns in parallel while being constrained by a fixed overall advertising budget. This gives rise to the problem of distributing the budget across the different campaigns dynamically so as to optimize the overall return on investment (ROI) (or some other metric) within a specified time duration. In this paper, we propose an RL formulation called BCQ4DCA for dynamic optimization of budget-constrained campaign allocation. The formulation is model-free and uses a novel cumulative reward model that is learned alongside a Deep Q-Network. We utilize a real-world Criteo user interaction dataset to evaluate BCQ4DCA in terms of conversion rate, budget utilization, cost per conversion, and ROI, finding that it outperforms current heuristic, attribution based approaches like DARNN, DNAMTA across a common time window.